You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/11/14 00:11:43 UTC
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1022
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1022/display/redirect>
------------------------------------------
[...truncated 235.53 KB...]
INFO: Adding Query7/Query7.Monitor as step s2
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 14, 2018 12:04:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88183 bytes, hash zI5v2VuVgSnvqD0I42iwgA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-zI5v2VuVgSnvqD0I42iwgA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 14, 2018 12:04:07 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-13_16_04_06-2153884204186820274
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-13_16_04_06-2153884204186820274?project=apache-beam-testing
Nov 14, 2018 12:04:07 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-13_16_04_06-2153884204186820274
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-14T00:04:06.426Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:09.047Z: Checking permissions granted to controller Service Account.
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:16.407Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:17.055Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:17.149Z: Expanding GroupByKey operations into optimizable parts.
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:17.199Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:17.403Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:17.452Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#7f4de336599327c8
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:17.504Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:17.550Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:17.595Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:17.636Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:17.686Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:17.729Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:17.777Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:17.814Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:17.861Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:17.915Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:17.960Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.012Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.060Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.108Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.157Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.204Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.252Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.289Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.336Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.383Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.420Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.468Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.519Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.567Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.612Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.654Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.795Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.892Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:18.951Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:19.029Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:19.077Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:19.123Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:19.613Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:19.664Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:19.708Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:19.720Z: Starting 4 workers in us-central1-b...
Nov 14, 2018 12:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:19.969Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 14, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:29.228Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 14, 2018 12:04:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:56.864Z: Workers have started successfully.
Nov 14, 2018 12:05:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:58.084Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 14, 2018 12:05:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:04:58.138Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 14, 2018 12:05:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:05:04.482Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 14, 2018 12:05:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:05:04.525Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 14, 2018 12:05:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:05:16.308Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 14, 2018 12:05:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:05:27.401Z: Workers have started successfully.
Nov 14, 2018 12:06:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:06:44.032Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 14, 2018 12:06:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:06:44.141Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 14, 2018 12:06:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:06:44.927Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 14, 2018 12:06:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:06:45.036Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 14, 2018 12:06:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:06:46.412Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 14, 2018 12:06:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:06:46.511Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 14, 2018 12:06:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:06:49.406Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 14, 2018 12:06:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:06:49.654Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 14, 2018 12:07:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:07:18.682Z: Cleaning up.
Nov 14, 2018 12:07:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:07:18.805Z: Stopping worker pool...
Nov 14, 2018 12:08:07 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-14T00:08:07.263Z Waiting for main pipeline to 'finish'WARNING: No terminal state was returned. State value RUNNING
2018-11-14T00:08:07.325Z RUNNING Query7
2018-11-14T00:08:08.462Z new perf {"snapshots":null,"runtimeSec":126.676,"numEvents":10000000,"eventsPerSec":78941.55167513972,"eventBytesPerSec":1.013266046449209E7,"numResults":100,"resultsPerSec":0.7894155167513972,"resultBytesPerSec":79.75464965739367,"startupDelaySec":-175.996,"processingDelaySec":100.89,"shutdownDelaySec":49.32,"timeDilation":7.76311219173324,"errors":null,"jobId":null}
2018-11-14T00:08:23.497Z RUNNING Query7
2018-11-14T00:08:24.509Z no activity
2018-11-14T00:08:39.557Z RUNNING Query7
2018-11-14T00:08:40.588Z no activity
2018-11-14T00:08:55.634Z RUNNING Query7
2018-11-14T00:08:56.693Z no activity
2018-11-14T00:09:11.730Z RUNNING Query7
2018-11-14T00:09:13.066Z no activity
2018-11-14T00:09:28.166Z RUNNING Query7
2018-11-14T00:09:29.159Z no activity
2018-11-14T00:09:44.194Z RUNNING Query7
2018-11-14T00:09:45.279Z no activity
2018-11-14T00:10:00.328Z RUNNING Query7
2018-11-14T00:10:01.404Z no activity
2018-11-14T00:10:16.996Z RUNNING Query7
2018-11-14T00:10:18.855Z no activity
2018-11-14T00:10:33.913Z RUNNING Query7
2018-11-14T00:10:35.644Z no activity
2018-11-14T00:10:50.686Z RUNNING Query7
2018-11-14T00:10:51.939Z no activity
2018-11-14T00:11:07.031Z RUNNING Query7
2018-11-14T00:11:08.077Z no activity
2018-11-14T00:11:23.131Z RUNNING Query7
2018-11-14T00:11:24.346Z no activity
2018-11-14T00:11:39.396Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 7,5,main]) completed. Took 7 mins 43.384 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 11m 33s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/oa2yy7hyhm3xo
Build cache (/home/jenkins/.gradle/caches/build-cache-1) removing files not accessed on or after Wed Nov 07 00:00:12 UTC 2018.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4337} MaxSize{4500}, CacheStats{hitCount=20, missCount=3984, loadSuccessCount=3985, loadExceptionCount=0, totalLoadTime=283499095, evictionCount=1}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Build cache (/home/jenkins/.gradle/caches/build-cache-1) cleaned up in 2.06 secs.
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_PostCommit_Java_Nexmark_Dataflow #1092
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1092/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1091
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1091/display/redirect?page=changes>
Changes:
[echauchot] [BEAM-5969] Fix typo in nexmark web page
[mxm] [website] Add 'builds' mailing list to the contact page
------------------------------------------
[...truncated 381.91 KB...]
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys as step s11
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) as step s12
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections as step s13
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView as step s14
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) as step s15
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/AsBid as step s16
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles as step s17
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Debug as step s18
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Stamp as step s19
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.Format as step s20
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.DevNull as step s21
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <58906 bytes, hash 4k0HKoRdW0mkWLHwM962Ng> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-4k0HKoRdW0mkWLHwM962Ng.pb
Nov 20, 2018 10:04:04 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88194 bytes, hash A5XVO8-mvoxNTyTEHhSaxw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-A5XVO8-mvoxNTyTEHhSaxw.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-20_02_04_05-9943991727944894115
Nov 20, 2018 10:04:06 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-20_02_04_05-9943991727944894115?project=apache-beam-testing
Nov 20, 2018 10:04:06 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-20_02_04_05-9943991727944894115
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-20T10:04:05.084Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:08.522Z: Checking permissions granted to controller Service Account.
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:13.439Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.154Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.238Z: Expanding GroupByKey operations into optimizable parts.
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.288Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.472Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.515Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#25398a999f74762b
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.557Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.608Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.656Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.695Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.730Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.768Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.814Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.861Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.896Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.937Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:14.982Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.030Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.083Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.126Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.165Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.201Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.239Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.288Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.339Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.383Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.417Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.468Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.511Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.543Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.585Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.632Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.666Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.715Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.758Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.800Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.849Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:15.887Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:16.407Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:16.446Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:16.484Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:16.517Z: Starting 4 workers in us-central1-b...
Nov 20, 2018 10:04:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:16.779Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 20, 2018 10:04:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:04:35.498Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 20, 2018 10:05:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:05:02.325Z: Workers have started successfully.
Nov 20, 2018 10:05:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:05:04.411Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 20, 2018 10:05:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:05:04.454Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 20, 2018 10:05:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:05:05.226Z: Workers have started successfully.
Nov 20, 2018 10:05:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:05:21.668Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 20, 2018 10:06:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:17.925Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 20, 2018 10:06:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:18.021Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 20, 2018 10:06:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:19.681Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 20, 2018 10:06:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:19.777Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 20, 2018 10:06:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:21.212Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 20, 2018 10:06:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:21.308Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 20, 2018 10:06:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:23.718Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 20, 2018 10:06:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:23.951Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 20, 2018 10:06:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:51.715Z: Cleaning up.
Nov 20, 2018 10:06:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T10:06:51.853Z: Stopping worker pool...
Nov 20, 2018 10:08:06 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-20T10:08:06.589Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-20T10:08:06.624Z RUNNING Query7
2018-11-20T10:08:07.781Z new perf {"snapshots":null,"runtimeSec":95.68,"numEvents":10000000,"eventsPerSec":104515.05016722408,"eventBytesPerSec":1.341518496028428E7,"numResults":100,"resultsPerSec":1.0451505016722407,"resultBytesPerSec":105.59155518394648,"startupDelaySec":-171.076,"processingDelaySec":70.044,"shutdownDelaySec":75.396,"timeDilation":10.278010033444815,"errors":null,"jobId":null}
2018-11-20T10:08:22.826Z RUNNING Query7
2018-11-20T10:08:24.042Z no activity
2018-11-20T10:08:39.080Z RUNNING Query7
2018-11-20T10:08:40.100Z no activity
2018-11-20T10:08:55.147Z RUNNING Query7
2018-11-20T10:08:56.208Z no activity
2018-11-20T10:09:11.264Z RUNNING Query7
2018-11-20T10:09:12.311Z no activity
2018-11-20T10:09:27.351Z RUNNING Query7
2018-11-20T10:09:28.445Z no activity
2018-11-20T10:09:43.486Z RUNNING Query7
2018-11-20T10:09:44.495Z no activity
2018-11-20T10:09:59.544Z RUNNING Query7
2018-11-20T10:09:59.705Z no activity
2018-11-20T10:10:14.706Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 10,5,main]) completed. Took 6 mins 39.386 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 55s
76 actionable tasks: 72 executed, 4 from cache
Publishing build scan...
https://gradle.com/s/6l2dekckqdki6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1090
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1090/display/redirect?page=changes>
Changes:
[relax] CoGroup transforms for schemas.
------------------------------------------
[...truncated 233.61 KB...]
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-SJ0iKljRwJqRaVO78HFu2A.jar
Nov 20, 2018 8:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-SJ0iKljRwJqRaVO78HFu2A.jar
Nov 20, 2018 8:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-SJ0iKljRwJqRaVO78HFu2A.jar
Nov 20, 2018 8:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-SJ0iKljRwJqRaVO78HFu2A.jar
Nov 20, 2018 8:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-SJ0iKljRwJqRaVO78HFu2A.jar
Nov 20, 2018 8:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-SJ0iKljRwJqRaVO78HFu2A.jar
Nov 20, 2018 8:50:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-SJ0iKljRwJqRaVO78HFu2A.jar
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 117 files cached, 29 files newly uploaded
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 20, 2018 8:50:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 20, 2018 8:50:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 20, 2018 8:50:25 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 20, 2018 8:50:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 20, 2018 8:50:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88179 bytes, hash a_H6UnUYycCHXUZUuqLt5Q> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-a_H6UnUYycCHXUZUuqLt5Q.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 20, 2018 8:50:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-20_00_50_25-6387333694799286878
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-20_00_50_25-6387333694799286878?project=apache-beam-testing
Nov 20, 2018 8:50:29 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-20_00_50_25-6387333694799286878
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-20T08:50:25.592Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:32.213Z: Checking permissions granted to controller Service Account.
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:38.389Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:38.872Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:38.945Z: Expanding GroupByKey operations into optimizable parts.
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:38.978Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.171Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.207Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.243Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.277Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.313Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.350Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.386Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.424Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.458Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.483Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.515Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.548Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.585Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.619Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.656Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.692Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.729Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.764Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.801Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.838Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.876Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.916Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.949Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:39.993Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.027Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.063Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.101Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.132Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.169Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.204Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.242Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.280Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.312Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.348Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.744Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.781Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.822Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:40.829Z: Starting 4 workers in us-central1-b...
Nov 20, 2018 8:50:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:41.083Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 20, 2018 8:50:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:50:53.500Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 20, 2018 8:51:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:51:23.338Z: Workers have started successfully.
Nov 20, 2018 8:51:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:51:27.376Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 20, 2018 8:51:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:51:27.418Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 20, 2018 8:51:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:51:30.461Z: Workers have started successfully.
Nov 20, 2018 8:51:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:51:39.489Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 20, 2018 8:51:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:51:39.522Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 20, 2018 8:51:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:51:45.228Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 20, 2018 8:52:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:52:44.749Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 20, 2018 8:52:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:52:44.868Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 20, 2018 8:52:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:52:46.726Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 20, 2018 8:52:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:52:46.843Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 20, 2018 8:52:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:52:49.417Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 20, 2018 8:52:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:52:49.539Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 20, 2018 8:52:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:52:53.090Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 20, 2018 8:52:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:52:53.340Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 20, 2018 8:53:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:53:18.672Z: Cleaning up.
Nov 20, 2018 8:53:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T08:53:18.778Z: Stopping worker pool...
Nov 20, 2018 8:54:30 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-20T08:54:30.056Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-20T08:54:30.094Z RUNNING Query7
2018-11-20T08:54:31.164Z new perf {"snapshots":null,"runtimeSec":101.228,"numEvents":10000000,"eventsPerSec":98786.89690599439,"eventBytesPerSec":1.2679939315209232E7,"numResults":100,"resultsPerSec":0.9878689690599439,"resultBytesPerSec":99.80440194412614,"startupDelaySec":-172.957,"processingDelaySec":77.905,"shutdownDelaySec":71.729,"timeDilation":9.71470344173549,"errors":null,"jobId":null}
2018-11-20T08:54:46.204Z RUNNING Query7
2018-11-20T08:54:47.255Z no activity
2018-11-20T08:55:02.287Z RUNNING Query7
2018-11-20T08:55:03.369Z no activity
2018-11-20T08:55:18.407Z RUNNING Query7
2018-11-20T08:55:19.524Z no activity
2018-11-20T08:55:34.561Z RUNNING Query7
2018-11-20T08:55:35.599Z no activity
2018-11-20T08:55:50.649Z RUNNING Query7
2018-11-20T08:55:51.681Z no activity
2018-11-20T08:56:06.736Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 3,5,main]) completed. Took 5 mins 49.985 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 35s
76 actionable tasks: 72 executed, 4 from cache
Publishing build scan...
https://gradle.com/s/zqqk2rkc4okkw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1089
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1089/display/redirect>
------------------------------------------
[...truncated 257.91 KB...]
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-vHRCqotCn6lvBiqXwBTqKQ.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:512)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:549)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 20, 2018 6:03:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-vHRCqotCn6lvBiqXwBTqKQ.jar
Nov 20, 2018 6:03:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-vHRCqotCn6lvBiqXwBTqKQ.jar
Nov 20, 2018 6:04:01 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 117 files cached, 29 files newly uploaded
Nov 20, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 20, 2018 6:04:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88182 bytes, hash w5soxkiSkC8m5e06Xa6Inw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-w5soxkiSkC8m5e06Xa6Inw.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 20, 2018 6:04:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-19_22_04_02-14667256735523976504
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-19_22_04_02-14667256735523976504?project=apache-beam-testing
Nov 20, 2018 6:04:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-19_22_04_02-14667256735523976504
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-20T06:04:02.741Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:05.777Z: Checking permissions granted to controller Service Account.
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:10.930Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:11.444Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:11.534Z: Expanding GroupByKey operations into optimizable parts.
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:11.573Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:11.773Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:11.822Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#a70ce18896339064
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:11.870Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:11.917Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:11.965Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.010Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.056Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.104Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.146Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.181Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.218Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.257Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.303Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.349Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.397Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.435Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.483Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.523Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.568Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.616Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.651Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.696Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.742Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.788Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.835Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.875Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.922Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:12.969Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.018Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.054Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.100Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.149Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.186Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.219Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.679Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.726Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.775Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:13.786Z: Starting 4 workers in us-central1-b...
Nov 20, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:14.095Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 20, 2018 6:04:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:33.168Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 20, 2018 6:04:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:56.598Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 20, 2018 6:04:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:04:56.634Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 20, 2018 6:05:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:05:02.650Z: Workers have started successfully.
Nov 20, 2018 6:05:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:05:02.694Z: Workers have started successfully.
Nov 20, 2018 6:05:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:05:14.624Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 20, 2018 6:06:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:19.742Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 20, 2018 6:06:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:19.846Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 20, 2018 6:06:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:20.624Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 20, 2018 6:06:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:20.728Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 20, 2018 6:06:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:22.462Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 20, 2018 6:06:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:22.564Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 20, 2018 6:06:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:25.841Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 20, 2018 6:06:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:26.031Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 20, 2018 6:06:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:54.183Z: Cleaning up.
Nov 20, 2018 6:06:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T06:06:54.334Z: Stopping worker pool...
Nov 20, 2018 6:08:03 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-20T06:08:03.694Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-20T06:08:03.728Z RUNNING Query7
2018-11-20T06:08:04.798Z new perf {"snapshots":null,"runtimeSec":96.918,"numEvents":10000000,"eventsPerSec":103180.00784168059,"eventBytesPerSec":1.3243823613776594E7,"numResults":100,"resultsPerSec":1.031800078416806,"resultBytesPerSec":104.2427619224499,"startupDelaySec":-167.159,"processingDelaySec":72.389,"shutdownDelaySec":70.241,"timeDilation":10.14672197115087,"errors":null,"jobId":null}
2018-11-20T06:08:19.844Z RUNNING Query7
2018-11-20T06:08:20.815Z no activity
2018-11-20T06:08:35.851Z RUNNING Query7
2018-11-20T06:08:36.929Z no activity
2018-11-20T06:08:51.969Z RUNNING Query7
2018-11-20T06:08:53.016Z no activity
2018-11-20T06:09:08.055Z RUNNING Query7
2018-11-20T06:09:09.152Z no activity
2018-11-20T06:09:24.189Z RUNNING Query7
2018-11-20T06:09:25.222Z no activity
2018-11-20T06:09:40.273Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 4,5,main]) completed. Took 6 mins 0.225 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 33s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/xqoamggzskfhs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1088
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1088/display/redirect?page=changes>
Changes:
[cushon] [BEAM-6096] illegal signature attribute when compiling with JDK 11
------------------------------------------
[...truncated 269.74 KB...]
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-JXk4_AkNyZ8UWLy68oYasg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-JXk4_AkNyZ8UWLy68oYasg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:432)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:287)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-JXk4_AkNyZ8UWLy68oYasg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-JXk4_AkNyZ8UWLy68oYasg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:512)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:549)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 20, 2018 5:51:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-JXk4_AkNyZ8UWLy68oYasg.jar
Nov 20, 2018 5:51:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-JXk4_AkNyZ8UWLy68oYasg.jar
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 117 files cached, 28 files newly uploaded
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3.ReadBounded as step s1
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3.Monitor as step s2
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3.Snoop as step s3
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Window.Into()/Window.Assign as step s4
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/justNewAuctions/IsNewAuction/ParDo(Anonymous) as step s5
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/justNewAuctions/AsAuction as step s6
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.InCategory/ParDo(Anonymous) as step s7
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/AuctionBySeller as step s8
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/justNewPersons/IsNewPerson/ParDo(Anonymous) as step s9
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/justNewPersons/AsPerson as step s10
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.InState/ParDo(Anonymous) as step s11
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/PersonById as step s12
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/CoGroupByKey/MakeUnionTable0 as step s13
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/CoGroupByKey/MakeUnionTable1 as step s14
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/CoGroupByKey/Flatten as step s15
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/CoGroupByKey/GBK as step s16
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/CoGroupByKey/ConstructCoGbkResultFn as step s17
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/ReifyWindows as step s18
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys as step s19
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.Join/ParDo(BatchStateful) as step s20
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.Project as step s21
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3.Debug as step s22
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3.Stamp as step s23
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3.Format as step s24
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3.DevNull as step s25
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 20, 2018 5:51:53 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <65289 bytes, hash E5tK16nmdPiAqXp-J5KmiA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-E5tK16nmdPiAqXp-J5KmiA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 20, 2018 5:51:55 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-19_21_51_54-3518111521880910996
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-19_21_51_54-3518111521880910996?project=apache-beam-testing
Nov 20, 2018 5:51:55 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-19_21_51_54-3518111521880910996
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-20T05:51:54.466Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:51:57.004Z: Checking permissions granted to controller Service Account.
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:03.349Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:03.997Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.096Z: Expanding GroupByKey operations into optimizable parts.
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.118Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.241Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.281Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#9a88db53ad38536b
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.316Z: Fusing unzipped copy of Query3/Query3/CoGroupByKey/GBK/Reify, through flatten Query3/Query3/CoGroupByKey/Flatten, into producer Query3/Query3/CoGroupByKey/MakeUnionTable0
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.351Z: Fusing consumer Query3/Query3.Stamp into Query3/Query3.Debug
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.388Z: Fusing consumer Query3.Format into Query3/Query3.Stamp
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.423Z: Unzipping flatten s15-u18 for input s16-reify-value0-c16
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.451Z: Fusing unzipped copy of Query3/Query3/CoGroupByKey/GBK/Write, through flatten s15-u18, into producer Query3/Query3/CoGroupByKey/GBK/Reify
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.484Z: Fusing consumer Query3.DevNull into Query3.Format
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.511Z: Fusing consumer Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Write into Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/ReifyWindows
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.548Z: Fusing consumer Query3/Query3/Query3.Project into Query3/Query3/Query3.Join/ParDo(BatchStateful)
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.586Z: Fusing consumer Query3/Query3.Debug into Query3/Query3/Query3.Project
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.618Z: Fusing consumer Query3/Query3/Query3.Join/ParDo(BatchStateful) into Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Read
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.656Z: Fusing consumer Query3/Query3/CoGroupByKey/GBK/GroupByWindow into Query3/Query3/CoGroupByKey/GBK/Read
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.693Z: Fusing consumer Query3/Query3/CoGroupByKey/ConstructCoGbkResultFn into Query3/Query3/CoGroupByKey/GBK/GroupByWindow
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.730Z: Fusing consumer Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/ReifyWindows into Query3/Query3/CoGroupByKey/ConstructCoGbkResultFn
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.769Z: Fusing consumer Query3/Query3/CoGroupByKey/GBK/Reify into Query3/Query3/CoGroupByKey/MakeUnionTable1
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.822Z: Fusing consumer Query3/Query3/CoGroupByKey/GBK/Write into Query3/Query3/CoGroupByKey/GBK/Reify
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.850Z: Fusing consumer Query3/Query3/justNewAuctions/IsNewAuction/ParDo(Anonymous) into Query3/Query3/Window.Into()/Window.Assign
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.892Z: Fusing consumer Query3/Query3/justNewPersons/IsNewPerson/ParDo(Anonymous) into Query3/Query3/Window.Into()/Window.Assign
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.926Z: Fusing consumer Query3/Query3/Query3.InCategory/ParDo(Anonymous) into Query3/Query3/justNewAuctions/AsAuction
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.961Z: Fusing consumer Query3/Query3.Snoop into Query3/Query3.Monitor
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:04.996Z: Fusing consumer Query3/Query3/PersonById into Query3/Query3/Query3.InState/ParDo(Anonymous)
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.032Z: Fusing consumer Query3/Query3/CoGroupByKey/MakeUnionTable1 into Query3/Query3/PersonById
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.072Z: Fusing consumer Query3/Query3/justNewAuctions/AsAuction into Query3/Query3/justNewAuctions/IsNewAuction/ParDo(Anonymous)
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.098Z: Fusing consumer Query3/Query3/Query3.InState/ParDo(Anonymous) into Query3/Query3/justNewPersons/AsPerson
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.134Z: Fusing consumer Query3/Query3/justNewPersons/AsPerson into Query3/Query3/justNewPersons/IsNewPerson/ParDo(Anonymous)
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.171Z: Fusing consumer Query3/Query3/AuctionBySeller into Query3/Query3/Query3.InCategory/ParDo(Anonymous)
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.208Z: Fusing consumer Query3/Query3.Monitor into Query3.ReadBounded
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.243Z: Fusing consumer Query3/Query3/Window.Into()/Window.Assign into Query3/Query3.Snoop
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.281Z: Fusing consumer Query3/Query3/CoGroupByKey/MakeUnionTable0 into Query3/Query3/AuctionBySeller
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.721Z: Executing operation Query3/Query3/CoGroupByKey/GBK/Create
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.766Z: Executing operation Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Create
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:05.818Z: Starting 4 workers in us-central1-b...
Nov 20, 2018 5:52:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:06.004Z: Executing operation Query3.ReadBounded+Query3/Query3.Monitor+Query3/Query3.Snoop+Query3/Query3/Window.Into()/Window.Assign+Query3/Query3/justNewAuctions/IsNewAuction/ParDo(Anonymous)+Query3/Query3/justNewPersons/IsNewPerson/ParDo(Anonymous)+Query3/Query3/justNewAuctions/AsAuction+Query3/Query3/Query3.InCategory/ParDo(Anonymous)+Query3/Query3/justNewPersons/AsPerson+Query3/Query3/Query3.InState/ParDo(Anonymous)+Query3/Query3/PersonById+Query3/Query3/CoGroupByKey/MakeUnionTable1+Query3/Query3/CoGroupByKey/GBK/Reify+Query3/Query3/AuctionBySeller+Query3/Query3/CoGroupByKey/MakeUnionTable0+Query3/Query3/CoGroupByKey/GBK/Reify+Query3/Query3/CoGroupByKey/GBK/Session/Flatten+Query3/Query3/CoGroupByKey/GBK/Write
Nov 20, 2018 5:52:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:22.985Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 20, 2018 5:52:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:46.717Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 20, 2018 5:52:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:46.764Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 20, 2018 5:52:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:49.530Z: Workers have started successfully.
Nov 20, 2018 5:52:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:49.568Z: Workers have started successfully.
Nov 20, 2018 5:52:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:58.722Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 20, 2018 5:52:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:52:58.762Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 20, 2018 5:53:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:53:05.127Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 20, 2018 5:53:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:53:44.084Z: Executing operation Query3/Query3/CoGroupByKey/GBK/Close
Nov 20, 2018 5:53:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:53:44.183Z: Executing operation Query3/Query3/CoGroupByKey/GBK/Read+Query3/Query3/CoGroupByKey/GBK/GroupByWindow+Query3/Query3/CoGroupByKey/ConstructCoGbkResultFn+Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/ReifyWindows+Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Write
Nov 20, 2018 5:53:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:53:50.103Z: Executing operation Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Close
Nov 20, 2018 5:53:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:53:50.200Z: Executing operation Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Read+Query3/Query3/Query3.Join/ParDo(BatchStateful)+Query3/Query3/Query3.Project+Query3/Query3.Debug+Query3/Query3.Stamp+Query3.Format+Query3.DevNull
Nov 20, 2018 5:53:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:53:53.892Z: Cleaning up.
Nov 20, 2018 5:53:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:53:53.997Z: Stopping worker pool...
Nov 20, 2018 5:55:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:55:53.597Z: Autoscaling: Resized worker pool from 4 to 0.
Nov 20, 2018 5:55:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T05:55:53.647Z: Worker pool stopped.
Nov 20, 2018 5:55:55 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-20T05:55:55.387Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-20T05:55:55.415Z RUNNING Query3
2018-11-20T05:55:56.633Z new perf {"snapshots":null,"runtimeSec":49.931,"numEvents":10000000,"eventsPerSec":200276.38140634075,"eventBytesPerSec":2.570677328713625E7,"numResults":59507,"resultsPerSec":1191.784662834712,"resultBytesPerSec":38085.918567623325,"startupDelaySec":-171.659,"processingDelaySec":47.128,"shutdownDelaySec":121.728,"timeDilation":-1.0,"errors":null,"jobId":null}
2018-11-20T05:56:11.690Z DONE Query3
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 3,5,main]) completed. Took 4 mins 46.303 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 8m 18s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/ko5qqku753iqa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1087
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1087/display/redirect>
------------------------------------------
[...truncated 240.14 KB...]
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-3MGJVaahA-SBw4a7T_wT-w.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:512)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:549)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 20, 2018 12:03:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-3MGJVaahA-SBw4a7T_wT-w.jar
Nov 20, 2018 12:04:00 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 117 files cached, 29 files newly uploaded
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 20, 2018 12:04:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88175 bytes, hash c0xVojXhSoNAj9385lvUdw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-c0xVojXhSoNAj9385lvUdw.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 20, 2018 12:04:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-19_16_04_02-11739024428374523017
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-19_16_04_02-11739024428374523017?project=apache-beam-testing
Nov 20, 2018 12:04:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-19_16_04_02-11739024428374523017
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-20T00:04:02.444Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:05.325Z: Checking permissions granted to controller Service Account.
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:12.471Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:13.114Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:13.213Z: Expanding GroupByKey operations into optimizable parts.
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:13.254Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:13.473Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:13.524Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#7f4de336599327c8
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:13.558Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:13.598Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:13.651Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:13.700Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:13.738Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:13.790Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:13.826Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:13.860Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:13.891Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:13.932Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:13.970Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:13.999Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.041Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.092Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.124Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.157Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.196Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.238Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.278Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.305Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.341Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.382Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.418Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.458Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.491Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.533Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.571Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.609Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.632Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.663Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.700Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:14.734Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:15.151Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:15.193Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:15.228Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:15.229Z: Starting 4 workers in us-central1-b...
Nov 20, 2018 12:04:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:15.501Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 20, 2018 12:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:25.611Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 20, 2018 12:04:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:48.487Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 20, 2018 12:04:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:48.529Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 20, 2018 12:04:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:51.125Z: Workers have started successfully.
Nov 20, 2018 12:04:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:04:51.223Z: Workers have started successfully.
Nov 20, 2018 12:05:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:05:05.891Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 20, 2018 12:05:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:05:05.921Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 20, 2018 12:05:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:05:12.083Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 20, 2018 12:06:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:06:11.970Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 20, 2018 12:06:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:06:12.094Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 20, 2018 12:06:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:06:13.613Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 20, 2018 12:06:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:06:13.715Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 20, 2018 12:06:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:06:15.052Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 20, 2018 12:06:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:06:15.156Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 20, 2018 12:06:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:06:18.774Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 20, 2018 12:06:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:06:18.940Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 20, 2018 12:06:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:06:48.120Z: Cleaning up.
Nov 20, 2018 12:06:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-20T00:06:48.325Z: Stopping worker pool...
Nov 20, 2018 12:08:03 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-20T00:08:03.376Z Waiting for main pipeline to 'finish'WARNING: No terminal state was returned. State value RUNNING
2018-11-20T00:08:03.430Z RUNNING Query7
2018-11-20T00:08:04.882Z new perf {"snapshots":null,"runtimeSec":103.491,"numEvents":10000000,"eventsPerSec":96626.75981486312,"eventBytesPerSec":1.2402671700920854E7,"numResults":100,"resultsPerSec":0.9662675981486313,"resultBytesPerSec":97.62201544095622,"startupDelaySec":-179.111,"processingDelaySec":76.143,"shutdownDelaySec":75.62,"timeDilation":9.50227556019364,"errors":null,"jobId":null}
2018-11-20T00:08:19.943Z RUNNING Query7
2018-11-20T00:08:20.979Z no activity
2018-11-20T00:08:36.022Z RUNNING Query7
2018-11-20T00:08:37.123Z no activity
2018-11-20T00:08:52.187Z RUNNING Query7
2018-11-20T00:08:53.397Z no activity
2018-11-20T00:09:08.447Z RUNNING Query7
2018-11-20T00:09:09.717Z no activity
2018-11-20T00:09:24.792Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 4,5,main]) completed. Took 5 mins 39.71 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 18s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/hhnnwmmdjm4zy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1086
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1086/display/redirect?page=changes>
Changes:
[amaliujia] [BEAM-6083] use same parsing functions to parse time
------------------------------------------
[...truncated 255.47 KB...]
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-vdAjqrH4BN5zUY_FaQ8Lpw.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:512)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:549)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 19, 2018 11:22:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-vdAjqrH4BN5zUY_FaQ8Lpw.jar
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 117 files cached, 29 files newly uploaded
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 19, 2018 11:22:34 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88172 bytes, hash g-nfkRBQXM_w88uHQO_-DQ> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-g-nfkRBQXM_w88uHQO_-DQ.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-19_15_22_35-247060295108997909
Nov 19, 2018 11:22:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-19_15_22_35-247060295108997909?project=apache-beam-testing
Nov 19, 2018 11:22:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-19_15_22_35-247060295108997909
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-19T23:22:35.334Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:38.443Z: Checking permissions granted to controller Service Account.
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:46.004Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:46.467Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:46.551Z: Expanding GroupByKey operations into optimizable parts.
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:46.596Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:46.747Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:46.779Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#84744397f8dc9244
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:46.806Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:46.835Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:46.877Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:46.908Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:46.941Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:46.979Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.008Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.043Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.076Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.104Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.140Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.177Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.214Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.251Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.288Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.328Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.361Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.398Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.426Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.457Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.494Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.529Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.565Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.599Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.621Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.652Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.678Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.713Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.740Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.771Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.807Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:47.844Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:48.223Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:48.245Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:48.281Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:48.297Z: Starting 4 workers in us-central1-b...
Nov 19, 2018 11:22:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:48.525Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 19, 2018 11:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:22:58.758Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 19, 2018 11:23:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:23:24.845Z: Workers have started successfully.
Nov 19, 2018 11:23:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:23:27.446Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 19, 2018 11:23:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:23:27.484Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 19, 2018 11:23:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:23:39.038Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 19, 2018 11:23:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:23:39.088Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 19, 2018 11:23:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:23:45.144Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 19, 2018 11:23:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:23:55.120Z: Workers have started successfully.
Nov 19, 2018 11:24:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:24:47.849Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 19, 2018 11:24:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:24:47.943Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 19, 2018 11:24:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:24:48.772Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 19, 2018 11:24:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:24:48.864Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 19, 2018 11:24:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:24:50.501Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 19, 2018 11:24:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:24:50.585Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 19, 2018 11:24:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:24:54.853Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 19, 2018 11:24:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:24:55.036Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 19, 2018 11:25:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:25:25.029Z: Cleaning up.
Nov 19, 2018 11:25:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T23:25:25.126Z: Stopping worker pool...
Nov 19, 2018 11:26:36 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-11-19T23:26:36.314Z Waiting for main pipeline to 'finish'
2018-11-19T23:26:36.372Z RUNNING Query7
2018-11-19T23:26:37.528Z new perf {"snapshots":null,"runtimeSec":105.712,"numEvents":10000000,"eventsPerSec":94596.63992734978,"eventBytesPerSec":1.2142092638489481E7,"numResults":100,"resultsPerSec":0.9459663992734978,"resultBytesPerSec":95.57098531860149,"startupDelaySec":-177.277,"processingDelaySec":77.988,"shutdownDelaySec":71.565,"timeDilation":9.302633570455576,"errors":null,"jobId":null}
2018-11-19T23:26:52.582Z RUNNING Query7
2018-11-19T23:26:53.767Z no activity
2018-11-19T23:27:08.806Z RUNNING Query7
2018-11-19T23:27:10.015Z no activity
2018-11-19T23:27:25.071Z RUNNING Query7
2018-11-19T23:27:26.405Z no activity
2018-11-19T23:27:41.450Z RUNNING Query7
2018-11-19T23:27:42.568Z no activity
2018-11-19T23:27:57.627Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 10,5,main]) completed. Took 5 mins 48.133 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 37s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/347rvz4fodqxo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1085
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1085/display/redirect?page=changes>
Changes:
[batbat] Upgraded apiary client version to 1.27.0.
------------------------------------------
[...truncated 266.06 KB...]
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-VzKLYIwPSK5cylYEBQRs5Q.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-VzKLYIwPSK5cylYEBQRs5Q.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:512)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:549)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 19, 2018 9:15:38 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-VzKLYIwPSK5cylYEBQRs5Q.jar
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 97 files cached, 49 files newly uploaded
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 19, 2018 9:15:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88191 bytes, hash RTS9wig63A0CD56uVO1N3w> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-RTS9wig63A0CD56uVO1N3w.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-19_13_15_43-773274572652502452
Nov 19, 2018 9:15:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-19_13_15_43-773274572652502452?project=apache-beam-testing
Nov 19, 2018 9:15:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-19_13_15_43-773274572652502452
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-19T21:15:43.184Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:45.875Z: Checking permissions granted to controller Service Account.
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:52.504Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:53.042Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:53.149Z: Expanding GroupByKey operations into optimizable parts.
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:53.185Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:53.382Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:53.426Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#37bc5e2e3faec20a
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:53.471Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:53.528Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:53.582Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:53.629Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:53.691Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:53.740Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:53.797Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:53.849Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:53.899Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:53.950Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.003Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.055Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.112Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.161Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.207Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.266Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.313Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.358Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.406Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.482Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.530Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.585Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.681Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.739Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.780Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.834Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.877Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.919Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:54.969Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:55.012Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:55.059Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:55.162Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:55.727Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:55.771Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:55.823Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:55.835Z: Starting 4 workers in us-central1-b...
Nov 19, 2018 9:16:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:15:56.125Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 19, 2018 9:16:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:16:08.234Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 19, 2018 9:16:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:16:42.200Z: Workers have started successfully.
Nov 19, 2018 9:16:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:16:49.206Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 19, 2018 9:17:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:17:12.375Z: Workers have started successfully.
Nov 19, 2018 9:18:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:17:59.383Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 19, 2018 9:18:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:17:59.488Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 19, 2018 9:18:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:18:00.203Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 19, 2018 9:18:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:18:00.291Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 19, 2018 9:18:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:18:02.727Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 19, 2018 9:18:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:18:02.808Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 19, 2018 9:18:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:18:05.805Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 19, 2018 9:18:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:18:06.001Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 19, 2018 9:18:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:18:33.122Z: Cleaning up.
Nov 19, 2018 9:18:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T21:18:33.233Z: Stopping worker pool...
Nov 19, 2018 9:19:44 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-19T21:19:44.023Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-19T21:19:44.070Z RUNNING Query7
2018-11-19T21:19:45.343Z new perf {"snapshots":null,"runtimeSec":96.596,"numEvents":10000000,"eventsPerSec":103523.95544328957,"eventBytesPerSec":1.3287971520559857E7,"numResults":100,"resultsPerSec":1.0352395544328958,"resultBytesPerSec":104.59025218435545,"startupDelaySec":-168.589,"processingDelaySec":72.101,"shutdownDelaySec":71.993,"timeDilation":10.180545778293096,"errors":null,"jobId":null}
2018-11-19T21:20:00.390Z RUNNING Query7
2018-11-19T21:20:01.740Z no activity
2018-11-19T21:20:16.838Z RUNNING Query7
2018-11-19T21:20:17.974Z no activity
2018-11-19T21:20:33.044Z RUNNING Query7
2018-11-19T21:20:34.434Z no activity
2018-11-19T21:20:49.522Z RUNNING Query7
2018-11-19T21:20:51.450Z no activity
2018-11-19T21:21:06.542Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 5,5,main]) completed. Took 5 mins 53.492 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 32s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/dtqwrnjnqkjwy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1084
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1084/display/redirect>
------------------------------------------
[...truncated 249.25 KB...]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-WrzthlNnJHVoj7ykTxMTaA.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-WrzthlNnJHVoj7ykTxMTaA.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 19, 2018 6:04:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-WrzthlNnJHVoj7ykTxMTaA.jar
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 19, 2018 6:04:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 19, 2018 6:04:14 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 19, 2018 6:04:14 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 19, 2018 6:04:14 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 19, 2018 6:04:14 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 19, 2018 6:04:14 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 19, 2018 6:04:14 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88185 bytes, hash cSFuJ1KF_a-sJn-dl71JYA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-cSFuJ1KF_a-sJn-dl71JYA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 19, 2018 6:04:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-19_10_04_14-17036458763679961965
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-19_10_04_14-17036458763679961965?project=apache-beam-testing
Nov 19, 2018 6:04:15 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-19_10_04_14-17036458763679961965
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-19T18:04:14.698Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:17.685Z: Checking permissions granted to controller Service Account.
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:24.140Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:24.606Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:24.691Z: Expanding GroupByKey operations into optimizable parts.
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:24.727Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:24.911Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:24.956Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#6044f02ac45a6f71
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.007Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.063Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.111Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.158Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.206Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.248Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.294Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.328Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.382Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.428Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.469Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.513Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.564Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.606Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.659Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.707Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.752Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.809Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.853Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.893Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.935Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:25.977Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:26.019Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:26.082Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:26.133Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:26.175Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:26.218Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:26.264Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:26.309Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:26.349Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:26.388Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:26.436Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:27.012Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:27.046Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:27.086Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:27.107Z: Starting 4 workers in us-central1-b...
Nov 19, 2018 6:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:27.428Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 19, 2018 6:04:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:04:39.022Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 19, 2018 6:05:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:05:08.422Z: Workers have started successfully.
Nov 19, 2018 6:05:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:05:19.722Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 19, 2018 6:05:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:05:38.667Z: Workers have started successfully.
Nov 19, 2018 6:06:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:06:27.323Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 19, 2018 6:06:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:06:27.401Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 19, 2018 6:06:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:06:29.043Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 19, 2018 6:06:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:06:29.132Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 19, 2018 6:06:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:06:31.605Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 19, 2018 6:06:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:06:31.673Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 19, 2018 6:06:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:06:34.969Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 19, 2018 6:06:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:06:35.165Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 19, 2018 6:07:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:07:00.055Z: Cleaning up.
Nov 19, 2018 6:07:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T18:07:00.152Z: Stopping worker pool...
Nov 19, 2018 6:08:15 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-19T18:08:15.805Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-19T18:08:15.841Z RUNNING Query7
2018-11-19T18:08:17.050Z new perf {"snapshots":null,"runtimeSec":95.301,"numEvents":10000000,"eventsPerSec":104930.69327709048,"eventBytesPerSec":1.3468535450834723E7,"numResults":100,"resultsPerSec":1.0493069327709048,"resultBytesPerSec":106.01147941784451,"startupDelaySec":-171.583,"processingDelaySec":73.218,"shutdownDelaySec":76.282,"timeDilation":10.318884376869077,"errors":null,"jobId":null}
2018-11-19T18:08:32.117Z RUNNING Query7
2018-11-19T18:08:33.227Z no activity
2018-11-19T18:08:48.287Z RUNNING Query7
2018-11-19T18:08:49.516Z no activity
2018-11-19T18:09:04.580Z RUNNING Query7
2018-11-19T18:09:05.880Z no activity
2018-11-19T18:09:20.920Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 2,5,main]) completed. Took 5 mins 29.315 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 14s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/6vbcrvzdrjwy2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1083
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1083/display/redirect?page=changes>
Changes:
[mxm] [BEAM-4681] Add support for portable timers in Flink batch mode
[mxm] [BEAM-4681] Use CoderUtils to decode key
[mxm] [BEAM-867] Support set/delete of timers by ID in FlinkRunner
[mxm] [BEAM-4681] Enable Java portable ValidatesRunner timer tests
[mxm] [BEAM-4681] Enable Python ValidatesRunner timer tests
[mxm] [BEAM-4681] Simplify by replacing the two timer keys with a single one
------------------------------------------
[...truncated 379.09 KB...]
Nov 19, 2018 4:09:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/MapElements/Map as step s3
Nov 19, 2018 4:09:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Monitor as step s4
Nov 19, 2018 4:09:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Snoop as step s5
Nov 19, 2018 4:09:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) as step s6
Nov 19, 2018 4:09:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s7
Nov 19, 2018 4:09:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) as step s8
Nov 19, 2018 4:09:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize as step s9
Nov 19, 2018 4:09:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) as step s10
Nov 19, 2018 4:09:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys as step s11
Nov 19, 2018 4:09:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) as step s12
Nov 19, 2018 4:09:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections as step s13
Nov 19, 2018 4:09:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView as step s14
Nov 19, 2018 4:09:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) as step s15
Nov 19, 2018 4:09:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/AsBid as step s16
Nov 19, 2018 4:09:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles as step s17
Nov 19, 2018 4:09:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Debug as step s18
Nov 19, 2018 4:09:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Stamp as step s19
Nov 19, 2018 4:09:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.Format as step s20
Nov 19, 2018 4:09:07 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.DevNull as step s21
Nov 19, 2018 4:09:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 19, 2018 4:09:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <58933 bytes, hash 1XV7Xl1Ldd0lKt6cqVtoDg> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-1XV7Xl1Ldd0lKt6cqVtoDg.pb
Nov 19, 2018 4:09:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-185dmBDT2hA3f6JXCdfhuQ.jar
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 19, 2018 4:09:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-185dmBDT2hA3f6JXCdfhuQ.jar
Nov 19, 2018 4:09:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-19_08_09_07-10491805752574827508
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-19_08_09_07-10491805752574827508?project=apache-beam-testing
Nov 19, 2018 4:09:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-19_08_09_07-10491805752574827508
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 117 files cached, 28 files newly uploaded
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 19, 2018 4:09:11 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88203 bytes, hash XIPCnsKzhin97x-Qt5FIyA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-XIPCnsKzhin97x-Qt5FIyA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-19T16:09:07.674Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:10.746Z: Checking permissions granted to controller Service Account.
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:15.768Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:16.374Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:16.533Z: Expanding GroupByKey operations into optimizable parts.
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:16.563Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:16.650Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:16.686Z: Fusing consumer JoinToFiles/JoinToFiles.Monitor into JoinToFiles.ReadBounded
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:16.721Z: Fusing consumer JoinToFiles/JoinToFiles.Snoop into JoinToFiles/JoinToFiles.Monitor
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:16.770Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) into JoinToFiles/JoinToFiles.Snoop
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:16.811Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/AsBid into JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:16.857Z: Fusing consumer JoinToFiles.Format into JoinToFiles/JoinToFiles.Stamp
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:16.892Z: Fusing consumer JoinToFiles.DevNull into JoinToFiles.Format
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:16.936Z: Fusing consumer JoinToFiles/JoinToFiles.Stamp into JoinToFiles/JoinToFiles.Debug
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:16.978Z: Fusing consumer JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles into JoinToFiles/JoinToFiles/justBids/AsBid
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:17.012Z: Fusing consumer JoinToFiles/JoinToFiles.Debug into JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:17.059Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:17.091Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:17.134Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:17.214Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) into NexmarkUtils.GenerateSideInputData/MapElements/Map
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:17.267Z: Fusing consumer NexmarkUtils.GenerateSideInputData/MapElements/Map into NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:17.313Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:17.354Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:17.388Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:17.906Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Create
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:17.961Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:18.008Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Create
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:18.020Z: Starting 4 workers in us-central1-b...
Nov 19, 2018 4:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:18.315Z: Executing operation NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)+NexmarkUtils.GenerateSideInputData/MapElements/Map+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 19, 2018 4:09:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:29.299Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 19, 2018 4:09:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:52.334Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 19, 2018 4:09:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:52.369Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 19, 2018 4:10:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:58.641Z: Workers have started successfully.
Nov 19, 2018 4:10:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:09:58.679Z: Workers have started successfully.
Nov 19, 2018 4:10:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:10:03.410Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 19, 2018 4:10:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:10:03.447Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 19, 2018 4:10:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:10:09.396Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 19, 2018 4:10:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:10:18.521Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 19, 2018 4:10:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:10:18.602Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write
Nov 19, 2018 4:10:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:10:24.990Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Close
Nov 19, 2018 4:10:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:10:25.021Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Close
Nov 19, 2018 4:10:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:10:25.052Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey)
Nov 19, 2018 4:10:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:10:25.095Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize)
Nov 19, 2018 4:10:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:10:31.633Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections
Nov 19, 2018 4:10:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:10:31.886Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView
Nov 19, 2018 4:10:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:10:32.073Z: Executing operation JoinToFiles.ReadBounded+JoinToFiles/JoinToFiles.Monitor+JoinToFiles/JoinToFiles.Snoop+JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)+JoinToFiles/JoinToFiles/justBids/AsBid+JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles+JoinToFiles/JoinToFiles.Debug+JoinToFiles/JoinToFiles.Stamp+JoinToFiles.Format+JoinToFiles.DevNull
Nov 19, 2018 4:11:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:11:28.316Z: Cleaning up.
Nov 19, 2018 4:11:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T16:11:28.436Z: Stopping worker pool...
Nov 19, 2018 4:13:08 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-19T16:13:08.390Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-19T16:13:08.429Z RUNNING JoinToFiles
2018-11-19T16:13:09.509Z new perf {"snapshots":null,"runtimeSec":53.893,"numEvents":10000000,"eventsPerSec":185552.85473067002,"eventBytesPerSec":2.3816913087042842E7,"numResults":9200000,"resultsPerSec":170708.62635221644,"resultBytesPerSec":6084868.071920287,"startupDelaySec":-154.18,"processingDelaySec":0.809,"shutdownDelaySec":100.287,"timeDilation":18.55435770879335,"errors":null,"jobId":null}
2018-11-19T16:13:24.545Z RUNNING JoinToFiles
2018-11-19T16:13:25.535Z no activity
2018-11-19T16:13:40.574Z RUNNING JoinToFiles
2018-11-19T16:13:41.597Z no activity
2018-11-19T16:13:56.636Z RUNNING JoinToFiles
2018-11-19T16:13:57.632Z no activity
2018-11-19T16:14:12.701Z DONE JoinToFiles
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 4,5,main]) completed. Took 5 mins 36.389 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 16s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/l4jmdtfgdeta4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1082
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1082/display/redirect?page=changes>
Changes:
[echauchot] [BEAM-5969] Add BOUNDED_SIDE_INPUT_JOIN new Query to Beam website
------------------------------------------
[...truncated 251.60 KB...]
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-TcaOL84XDQbz_qXK48pSNg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-TcaOL84XDQbz_qXK48pSNg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 19, 2018 3:25:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-TcaOL84XDQbz_qXK48pSNg.jar
Nov 19, 2018 3:26:00 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 19, 2018 3:26:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88176 bytes, hash IoDa3VhZjVHkM_WjV5P3dA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-IoDa3VhZjVHkM_WjV5P3dA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-19_07_26_01-2613775342329271758
Nov 19, 2018 3:26:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-19_07_26_01-2613775342329271758?project=apache-beam-testing
Nov 19, 2018 3:26:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-19_07_26_01-2613775342329271758
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-19T15:26:01.817Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:04.421Z: Checking permissions granted to controller Service Account.
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:10.132Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:10.615Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:10.696Z: Expanding GroupByKey operations into optimizable parts.
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:10.750Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:10.945Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:10.984Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#6044f02ac45a6f71
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.034Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.068Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.112Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.152Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.194Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.242Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.290Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.333Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.366Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.410Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.449Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.497Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.532Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.564Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.604Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.640Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.664Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.707Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.755Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.792Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.830Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.868Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.916Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:11.961Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:12.010Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:12.058Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:12.105Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:12.143Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:12.177Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:12.217Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:12.254Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:12.290Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:12.715Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:12.754Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:12.786Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:12.812Z: Starting 4 workers in us-central1-b...
Nov 19, 2018 3:26:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:13.044Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 19, 2018 3:26:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:23.714Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 19, 2018 3:26:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:49.478Z: Workers have started successfully.
Nov 19, 2018 3:26:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:52.427Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 19, 2018 3:26:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:52.463Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 19, 2018 3:26:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:56.432Z: Workers have started successfully.
Nov 19, 2018 3:27:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:58.259Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 19, 2018 3:27:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:26:58.307Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 19, 2018 3:27:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:27:09.491Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 19, 2018 3:28:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:28:09.841Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 19, 2018 3:28:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:28:09.924Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 19, 2018 3:28:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:28:11.476Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 19, 2018 3:28:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:28:11.553Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 19, 2018 3:28:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:28:12.906Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 19, 2018 3:28:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:28:12.976Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 19, 2018 3:28:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:28:16.639Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 19, 2018 3:28:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:28:16.811Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 19, 2018 3:28:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:28:42.705Z: Cleaning up.
Nov 19, 2018 3:28:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T15:28:42.803Z: Stopping worker pool...
Nov 19, 2018 3:30:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-19T15:30:02.676Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-19T15:30:02.720Z RUNNING Query7
2018-11-19T15:30:03.842Z new perf {"snapshots":null,"runtimeSec":98.609,"numEvents":10000000,"eventsPerSec":101410.62174852194,"eventBytesPerSec":1.3016711425934752E7,"numResults":100,"resultsPerSec":1.0141062174852196,"resultBytesPerSec":102.45515115253173,"startupDelaySec":-179.357,"processingDelaySec":75.364,"shutdownDelaySec":80.748,"timeDilation":9.972720542749649,"errors":null,"jobId":null}
2018-11-19T15:30:18.881Z RUNNING Query7
2018-11-19T15:30:19.879Z no activity
2018-11-19T15:30:34.941Z RUNNING Query7
2018-11-19T15:30:36.015Z no activity
2018-11-19T15:30:51.080Z RUNNING Query7
2018-11-19T15:30:52.179Z no activity
2018-11-19T15:31:07.238Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':',5,main]) completed. Took 5 mins 36.231 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 23s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/oecadoswdcwik
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1081
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1081/display/redirect?page=changes>
Changes:
[robbe.sneyders] Fix unorderable types in Python 3
[robbe.sneyders] Check equality of collections by sorting first, fall back to slower
------------------------------------------
[...truncated 270.81 KB...]
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-LP8Z5DZBYSls1Ii6SrBICQ.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-LP8Z5DZBYSls1Ii6SrBICQ.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 19, 2018 1:03:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-LP8Z5DZBYSls1Ii6SrBICQ.jar
Nov 19, 2018 1:03:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-LP8Z5DZBYSls1Ii6SrBICQ.jar
Nov 19, 2018 1:03:08 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 19, 2018 1:03:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88175 bytes, hash cjr_xFNiODh5fwiFrxbdfQ> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-cjr_xFNiODh5fwiFrxbdfQ.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 19, 2018 1:03:10 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-19_05_03_09-12742929081357882146?project=apache-beam-testing
Submitted job: 2018-11-19_05_03_09-12742929081357882146
Nov 19, 2018 1:03:10 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-19_05_03_09-12742929081357882146
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-19T13:03:09.798Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:12.721Z: Checking permissions granted to controller Service Account.
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:17.196Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:17.689Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:17.774Z: Expanding GroupByKey operations into optimizable parts.
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:17.860Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.056Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.102Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.151Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.197Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.243Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.291Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.339Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.388Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.432Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.474Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.513Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.557Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.594Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.642Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.689Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.738Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.785Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.831Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.879Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.923Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:18.968Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:19.027Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:19.066Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:19.103Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:19.146Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:19.196Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:19.230Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:19.260Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:19.303Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:19.369Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:19.481Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:19.546Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:19.637Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:19.699Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:20.203Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:20.251Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:20.297Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:20.309Z: Starting 4 workers in us-central1-b...
Nov 19, 2018 1:03:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:20.599Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 19, 2018 1:03:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:29.508Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 19, 2018 1:03:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:56.063Z: Workers have started successfully.
Nov 19, 2018 1:03:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:57.429Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 19, 2018 1:03:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:03:57.471Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 19, 2018 1:04:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:04:09.661Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 19, 2018 1:04:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:04:26.258Z: Workers have started successfully.
Nov 19, 2018 1:05:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:05:12.306Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 19, 2018 1:05:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:05:12.422Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 19, 2018 1:05:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:05:14.145Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 19, 2018 1:05:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:05:14.251Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 19, 2018 1:05:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:05:14.684Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 19, 2018 1:05:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:05:14.769Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 19, 2018 1:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:05:17.934Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 19, 2018 1:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:05:18.131Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 19, 2018 1:05:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:05:40.665Z: Cleaning up.
Nov 19, 2018 1:05:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T13:05:40.806Z: Stopping worker pool...
Nov 19, 2018 1:07:10 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-19T13:07:10.471Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-19T13:07:10.515Z RUNNING Query7
2018-11-19T13:07:11.622Z new perf {"snapshots":null,"runtimeSec":90.665,"numEvents":10000000,"eventsPerSec":110296.145149727,"eventBytesPerSec":1.415722601886064E7,"numResults":100,"resultsPerSec":1.10296145149727,"resultBytesPerSec":111.43219544476919,"startupDelaySec":-180.886,"processingDelaySec":70.353,"shutdownDelaySec":90.221,"timeDilation":10.846522914024154,"errors":null,"jobId":null}
2018-11-19T13:07:26.667Z RUNNING Query7
2018-11-19T13:07:27.665Z no activity
2018-11-19T13:07:42.700Z RUNNING Query7
2018-11-19T13:07:43.678Z no activity
2018-11-19T13:07:58.720Z RUNNING Query7
2018-11-19T13:07:59.791Z no activity
2018-11-19T13:08:14.827Z RUNNING Query7
2018-11-19T13:08:15.882Z no activity
2018-11-19T13:08:30.922Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':',5,main]) completed. Took 5 mins 49.95 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 23s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/7f63pogk7ireq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1080
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1080/display/redirect>
------------------------------------------
[...truncated 260.00 KB...]
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-AMcqIIX7HBb2ui2k_IuGwA.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-AMcqIIX7HBb2ui2k_IuGwA.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Nov 19, 2018 12:04:55 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar>
java.io.IOException: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-AMcqIIX7HBb2ui2k_IuGwA.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-AMcqIIX7HBb2ui2k_IuGwA.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:432)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:287)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-AMcqIIX7HBb2ui2k_IuGwA.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-AMcqIIX7HBb2ui2k_IuGwA.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 19, 2018 12:05:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-AMcqIIX7HBb2ui2k_IuGwA.jar
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.ReadBounded as step s1
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource) as step s2
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/MapElements/Map as step s3
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Monitor as step s4
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Snoop as step s5
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) as step s6
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s7
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) as step s8
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize as step s9
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) as step s10
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys as step s11
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) as step s12
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections as step s13
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView as step s14
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) as step s15
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/AsBid as step s16
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles as step s17
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Debug as step s18
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Stamp as step s19
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.Format as step s20
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.DevNull as step s21
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 19, 2018 12:05:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <58918 bytes, hash XCOGjbMs4ILUNzTZP8b-AQ> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-XCOGjbMs4ILUNzTZP8b-AQ.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 19, 2018 12:05:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-19_04_05_07-4988522667580810347
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-19_04_05_07-4988522667580810347?project=apache-beam-testing
Nov 19, 2018 12:05:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-19_04_05_07-4988522667580810347
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-19T12:05:07.348Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:10.099Z: Checking permissions granted to controller Service Account.
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:15.816Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:16.299Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:16.494Z: Expanding GroupByKey operations into optimizable parts.
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:16.526Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:16.617Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:16.656Z: Fusing consumer JoinToFiles/JoinToFiles.Monitor into JoinToFiles.ReadBounded
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:16.704Z: Fusing consumer JoinToFiles/JoinToFiles.Snoop into JoinToFiles/JoinToFiles.Monitor
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:16.753Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) into JoinToFiles/JoinToFiles.Snoop
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:16.802Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/AsBid into JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:16.853Z: Fusing consumer JoinToFiles.Format into JoinToFiles/JoinToFiles.Stamp
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:16.895Z: Fusing consumer JoinToFiles.DevNull into JoinToFiles.Format
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:16.943Z: Fusing consumer JoinToFiles/JoinToFiles.Stamp into JoinToFiles/JoinToFiles.Debug
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:16.984Z: Fusing consumer JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles into JoinToFiles/JoinToFiles/justBids/AsBid
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:17.023Z: Fusing consumer JoinToFiles/JoinToFiles.Debug into JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:17.069Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:17.118Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:17.162Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:17.209Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) into NexmarkUtils.GenerateSideInputData/MapElements/Map
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:17.254Z: Fusing consumer NexmarkUtils.GenerateSideInputData/MapElements/Map into NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:17.284Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:17.329Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:17.372Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:17.866Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Create
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:17.907Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:17.946Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Create
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:17.981Z: Starting 4 workers in us-central1-b...
Nov 19, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:18.229Z: Executing operation NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)+NexmarkUtils.GenerateSideInputData/MapElements/Map+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 19, 2018 12:05:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:05:39.139Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 19, 2018 12:06:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:06:08.405Z: Workers have started successfully.
Nov 19, 2018 12:06:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:06:12.705Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 19, 2018 12:06:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:06:12.738Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 19, 2018 12:06:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:06:18.212Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 19, 2018 12:06:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:06:27.341Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 19, 2018 12:06:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:06:27.479Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write
Nov 19, 2018 12:06:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:06:33.710Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Close
Nov 19, 2018 12:06:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:06:33.761Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Close
Nov 19, 2018 12:06:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:06:33.811Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey)
Nov 19, 2018 12:06:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:06:33.861Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize)
Nov 19, 2018 12:06:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:06:39.566Z: Workers have started successfully.
Nov 19, 2018 12:06:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:06:42.807Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections
Nov 19, 2018 12:06:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:06:43.124Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView
Nov 19, 2018 12:06:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:06:43.384Z: Executing operation JoinToFiles.ReadBounded+JoinToFiles/JoinToFiles.Monitor+JoinToFiles/JoinToFiles.Snoop+JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)+JoinToFiles/JoinToFiles/justBids/AsBid+JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles+JoinToFiles/JoinToFiles.Debug+JoinToFiles/JoinToFiles.Stamp+JoinToFiles.Format+JoinToFiles.DevNull
Nov 19, 2018 12:07:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:07:42.395Z: Cleaning up.
Nov 19, 2018 12:07:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T12:07:42.528Z: Stopping worker pool...
Nov 19, 2018 12:09:08 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-19T12:09:08.249Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-19T12:09:08.280Z RUNNING JoinToFiles
2018-11-19T12:09:09.368Z new perf {"snapshots":null,"runtimeSec":56.567,"numEvents":10000000,"eventsPerSec":176781.51572471583,"eventBytesPerSec":2.2691054802269876E7,"numResults":9200000,"resultsPerSec":162638.99446673857,"resultBytesPerSec":5797227.977442679,"startupDelaySec":-143.625,"processingDelaySec":0.86,"shutdownDelaySec":87.058,"timeDilation":17.67726766489296,"errors":null,"jobId":null}
2018-11-19T12:09:24.418Z RUNNING JoinToFiles
2018-11-19T12:09:25.580Z no activity
2018-11-19T12:09:40.623Z RUNNING JoinToFiles
2018-11-19T12:09:41.684Z no activity
2018-11-19T12:09:56.726Z RUNNING JoinToFiles
2018-11-19T12:09:57.854Z no activity
2018-11-19T12:10:12.898Z DONE JoinToFiles
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Daemon worker,5,main]) completed. Took 5 mins 34.205 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 57s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/jbl77xqfxg2w6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1079
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1079/display/redirect?page=changes>
Changes:
[robertwb] Implement profiling for Python DirectRunner and SDK Harness.
------------------------------------------
[...truncated 275.39 KB...]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Nov 19, 2018 10:01:49 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar>
java.io.IOException: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-TayvjSPHqqmk9WZIHinJhg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-TayvjSPHqqmk9WZIHinJhg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:432)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:287)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-TayvjSPHqqmk9WZIHinJhg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-TayvjSPHqqmk9WZIHinJhg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 19, 2018 10:01:55 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-TayvjSPHqqmk9WZIHinJhg.jar
Nov 19, 2018 10:01:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-TayvjSPHqqmk9WZIHinJhg.jar
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3.ReadBounded as step s1
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3.Monitor as step s2
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3.Snoop as step s3
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Window.Into()/Window.Assign as step s4
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/justNewAuctions/IsNewAuction/ParDo(Anonymous) as step s5
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/justNewAuctions/AsAuction as step s6
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.InCategory/ParDo(Anonymous) as step s7
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/AuctionBySeller as step s8
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/justNewPersons/IsNewPerson/ParDo(Anonymous) as step s9
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/justNewPersons/AsPerson as step s10
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.InState/ParDo(Anonymous) as step s11
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/PersonById as step s12
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/CoGroupByKey/MakeUnionTable0 as step s13
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/CoGroupByKey/MakeUnionTable1 as step s14
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/CoGroupByKey/Flatten as step s15
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/CoGroupByKey/GBK as step s16
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/CoGroupByKey/ConstructCoGbkResultFn as step s17
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/ReifyWindows as step s18
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys as step s19
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.Join/ParDo(BatchStateful) as step s20
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3/Query3.Project as step s21
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3.Debug as step s22
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3/Query3.Stamp as step s23
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3.Format as step s24
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query3.DevNull as step s25
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 19, 2018 10:02:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <65272 bytes, hash grKeLsdL3_hUklqH3hU2lQ> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-grKeLsdL3_hUklqH3hU2lQ.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 19, 2018 10:02:02 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-19_02_02_02-16030366142429415685
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-19_02_02_02-16030366142429415685?project=apache-beam-testing
Nov 19, 2018 10:02:02 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-19_02_02_02-16030366142429415685
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-19T10:02:02.237Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:04.883Z: Checking permissions granted to controller Service Account.
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:08.651Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:09.146Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:09.318Z: Expanding GroupByKey operations into optimizable parts.
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:09.363Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:09.520Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:09.572Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#bed3177b52ae68e
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:09.612Z: Fusing unzipped copy of Query3/Query3/CoGroupByKey/GBK/Reify, through flatten Query3/Query3/CoGroupByKey/Flatten, into producer Query3/Query3/CoGroupByKey/MakeUnionTable0
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:09.665Z: Fusing consumer Query3/Query3.Stamp into Query3/Query3.Debug
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:09.713Z: Fusing consumer Query3.Format into Query3/Query3.Stamp
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:09.774Z: Unzipping flatten s15-u18 for input s16-reify-value0-c16
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:09.822Z: Fusing unzipped copy of Query3/Query3/CoGroupByKey/GBK/Write, through flatten s15-u18, into producer Query3/Query3/CoGroupByKey/GBK/Reify
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:09.874Z: Fusing consumer Query3.DevNull into Query3.Format
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:09.908Z: Fusing consumer Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Write into Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/ReifyWindows
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:09.960Z: Fusing consumer Query3/Query3/Query3.Project into Query3/Query3/Query3.Join/ParDo(BatchStateful)
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:09.998Z: Fusing consumer Query3/Query3.Debug into Query3/Query3/Query3.Project
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.061Z: Fusing consumer Query3/Query3/Query3.Join/ParDo(BatchStateful) into Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Read
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.120Z: Fusing consumer Query3/Query3/CoGroupByKey/GBK/GroupByWindow into Query3/Query3/CoGroupByKey/GBK/Read
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.161Z: Fusing consumer Query3/Query3/CoGroupByKey/ConstructCoGbkResultFn into Query3/Query3/CoGroupByKey/GBK/GroupByWindow
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.214Z: Fusing consumer Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/ReifyWindows into Query3/Query3/CoGroupByKey/ConstructCoGbkResultFn
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.261Z: Fusing consumer Query3/Query3/CoGroupByKey/GBK/Reify into Query3/Query3/CoGroupByKey/MakeUnionTable1
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.298Z: Fusing consumer Query3/Query3/CoGroupByKey/GBK/Write into Query3/Query3/CoGroupByKey/GBK/Reify
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.346Z: Fusing consumer Query3/Query3/justNewAuctions/IsNewAuction/ParDo(Anonymous) into Query3/Query3/Window.Into()/Window.Assign
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.392Z: Fusing consumer Query3/Query3/justNewPersons/IsNewPerson/ParDo(Anonymous) into Query3/Query3/Window.Into()/Window.Assign
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.439Z: Fusing consumer Query3/Query3/Query3.InCategory/ParDo(Anonymous) into Query3/Query3/justNewAuctions/AsAuction
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.486Z: Fusing consumer Query3/Query3.Snoop into Query3/Query3.Monitor
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.526Z: Fusing consumer Query3/Query3/PersonById into Query3/Query3/Query3.InState/ParDo(Anonymous)
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.572Z: Fusing consumer Query3/Query3/CoGroupByKey/MakeUnionTable1 into Query3/Query3/PersonById
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.621Z: Fusing consumer Query3/Query3/justNewAuctions/AsAuction into Query3/Query3/justNewAuctions/IsNewAuction/ParDo(Anonymous)
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.668Z: Fusing consumer Query3/Query3/Query3.InState/ParDo(Anonymous) into Query3/Query3/justNewPersons/AsPerson
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.708Z: Fusing consumer Query3/Query3/justNewPersons/AsPerson into Query3/Query3/justNewPersons/IsNewPerson/ParDo(Anonymous)
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.755Z: Fusing consumer Query3/Query3/AuctionBySeller into Query3/Query3/Query3.InCategory/ParDo(Anonymous)
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.803Z: Fusing consumer Query3/Query3.Monitor into Query3.ReadBounded
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.851Z: Fusing consumer Query3/Query3/Window.Into()/Window.Assign into Query3/Query3.Snoop
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:10.900Z: Fusing consumer Query3/Query3/CoGroupByKey/MakeUnionTable0 into Query3/Query3/AuctionBySeller
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:11.416Z: Executing operation Query3/Query3/CoGroupByKey/GBK/Create
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:11.464Z: Executing operation Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Create
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:11.512Z: Starting 4 workers in us-central1-b...
Nov 19, 2018 10:02:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:11.741Z: Executing operation Query3.ReadBounded+Query3/Query3.Monitor+Query3/Query3.Snoop+Query3/Query3/Window.Into()/Window.Assign+Query3/Query3/justNewAuctions/IsNewAuction/ParDo(Anonymous)+Query3/Query3/justNewPersons/IsNewPerson/ParDo(Anonymous)+Query3/Query3/justNewAuctions/AsAuction+Query3/Query3/Query3.InCategory/ParDo(Anonymous)+Query3/Query3/justNewPersons/AsPerson+Query3/Query3/Query3.InState/ParDo(Anonymous)+Query3/Query3/PersonById+Query3/Query3/CoGroupByKey/MakeUnionTable1+Query3/Query3/CoGroupByKey/GBK/Reify+Query3/Query3/AuctionBySeller+Query3/Query3/CoGroupByKey/MakeUnionTable0+Query3/Query3/CoGroupByKey/GBK/Reify+Query3/Query3/CoGroupByKey/GBK/Session/Flatten+Query3/Query3/CoGroupByKey/GBK/Write
Nov 19, 2018 10:02:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:23.587Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 19, 2018 10:02:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:49.455Z: Workers have started successfully.
Nov 19, 2018 10:02:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:02:57.862Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 19, 2018 10:03:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:03:19.882Z: Workers have started successfully.
Nov 19, 2018 10:03:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:03:43.983Z: Executing operation Query3/Query3/CoGroupByKey/GBK/Close
Nov 19, 2018 10:03:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:03:44.061Z: Executing operation Query3/Query3/CoGroupByKey/GBK/Read+Query3/Query3/CoGroupByKey/GBK/GroupByWindow+Query3/Query3/CoGroupByKey/ConstructCoGbkResultFn+Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/ReifyWindows+Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Write
Nov 19, 2018 10:03:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:03:48.586Z: Executing operation Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Close
Nov 19, 2018 10:03:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:03:48.668Z: Executing operation Query3/Query3/Query3.Join/BatchStatefulParDoOverrides.GbkBeforeStatefulParDo/PartitionKeys/Read+Query3/Query3/Query3.Join/ParDo(BatchStateful)+Query3/Query3/Query3.Project+Query3/Query3.Debug+Query3/Query3.Stamp+Query3.Format+Query3.DevNull
Nov 19, 2018 10:03:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:03:53.058Z: Cleaning up.
Nov 19, 2018 10:03:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:03:53.138Z: Stopping worker pool...
Nov 19, 2018 10:05:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:05:46.519Z: Autoscaling: Resized worker pool from 4 to 0.
Nov 19, 2018 10:05:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T10:05:46.573Z: Worker pool stopped.
Nov 19, 2018 10:05:53 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
INFO: Job 2018-11-19_02_02_02-16030366142429415685 finished with status DONE.
2018-11-19T10:05:53.716Z Waiting for main pipeline to 'finish'
2018-11-19T10:05:53.716Z DONE Query3
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 6,5,main]) completed. Took 4 mins 20.904 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 7m 46s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/dwia4atd3lblo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1078
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1078/display/redirect?page=changes>
Changes:
[lukasz.gajowy] [BEAM-6076] Fetch only required fields for BigQuery Table
------------------------------------------
[...truncated 276.82 KB...]
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-OBgJSP2NNoyihGY9r3ESWg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 19, 2018 9:52:25 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-OBgJSP2NNoyihGY9r3ESWg.jar
Nov 19, 2018 9:52:28 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-OBgJSP2NNoyihGY9r3ESWg.jar
Nov 19, 2018 9:52:31 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-OBgJSP2NNoyihGY9r3ESWg.jar
Nov 19, 2018 9:52:33 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 19, 2018 9:52:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 19, 2018 9:52:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 19, 2018 9:52:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 19, 2018 9:52:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 19, 2018 9:52:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 19, 2018 9:52:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 19, 2018 9:52:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 19, 2018 9:52:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 19, 2018 9:52:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 19, 2018 9:52:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 19, 2018 9:52:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 19, 2018 9:52:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 19, 2018 9:52:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 19, 2018 9:52:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 19, 2018 9:52:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 19, 2018 9:52:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 19, 2018 9:52:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 19, 2018 9:52:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 19, 2018 9:52:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 19, 2018 9:52:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 19, 2018 9:52:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 19, 2018 9:52:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 19, 2018 9:52:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 19, 2018 9:52:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 19, 2018 9:52:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 19, 2018 9:52:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 19, 2018 9:52:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 19, 2018 9:52:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 19, 2018 9:52:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88177 bytes, hash oMgfbu2Q-xjgi7TAYwgSSg> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-oMgfbu2Q-xjgi7TAYwgSSg.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 19, 2018 9:52:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-19_01_52_34-7362580033603177336
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-19_01_52_34-7362580033603177336?project=apache-beam-testing
Nov 19, 2018 9:52:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-19_01_52_34-7362580033603177336
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-19T09:52:34.588Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:37.306Z: Checking permissions granted to controller Service Account.
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:44.093Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:44.626Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:44.703Z: Expanding GroupByKey operations into optimizable parts.
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:44.739Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:44.898Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:44.937Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:44.976Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:44.999Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.031Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.069Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.111Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.148Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.184Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.224Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.258Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.298Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.336Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.373Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.413Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.448Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.500Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.536Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.571Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.603Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.641Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.678Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.716Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.756Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.783Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.812Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.850Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.886Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.925Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.962Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:45.999Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:46.037Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:46.065Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:46.101Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:46.513Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:46.541Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:46.579Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:46.589Z: Starting 4 workers in us-central1-b...
Nov 19, 2018 9:52:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:46.814Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 19, 2018 9:52:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:52:58.574Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 19, 2018 9:53:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:53:24.552Z: Workers have started successfully.
Nov 19, 2018 9:53:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:53:27.538Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 19, 2018 9:53:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:53:27.571Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 19, 2018 9:53:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:53:32.139Z: Workers have started successfully.
Nov 19, 2018 9:53:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:53:33.457Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 19, 2018 9:54:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:54:46.404Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 19, 2018 9:54:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:54:46.485Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 19, 2018 9:54:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:54:48.086Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 19, 2018 9:54:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:54:48.156Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 19, 2018 9:54:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:54:49.599Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 19, 2018 9:54:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:54:49.670Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 19, 2018 9:54:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:54:52.718Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 19, 2018 9:54:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:54:52.891Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 19, 2018 9:55:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:55:16.712Z: Cleaning up.
Nov 19, 2018 9:55:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T09:55:16.814Z: Stopping worker pool...
Nov 19, 2018 9:56:35 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-11-19T09:56:35.402Z Waiting for main pipeline to 'finish'
2018-11-19T09:56:35.440Z RUNNING Query7
2018-11-19T09:56:36.469Z new perf {"snapshots":null,"runtimeSec":98.223,"numEvents":10000000,"eventsPerSec":101809.1485700905,"eventBytesPerSec":1.3067864929802593E7,"numResults":100,"resultsPerSec":1.0180914857009051,"resultBytesPerSec":102.85778280036244,"startupDelaySec":-177.32,"processingDelaySec":76.336,"shutdownDelaySec":79.097,"timeDilation":10.0119116703827,"errors":null,"jobId":null}
2018-11-19T09:56:51.510Z RUNNING Query7
2018-11-19T09:56:52.556Z no activity
2018-11-19T09:57:07.597Z RUNNING Query7
2018-11-19T09:57:08.549Z no activity
2018-11-19T09:57:23.585Z RUNNING Query7
2018-11-19T09:57:24.481Z no activity
2018-11-19T09:57:39.510Z RUNNING Query7
2018-11-19T09:57:40.482Z no activity
2018-11-19T09:57:55.536Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 7,5,main]) completed. Took 5 mins 48.081 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 10s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/wcay2shgzh2ce
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1077
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1077/display/redirect>
------------------------------------------
[...truncated 249.53 KB...]
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-v08U42TZqPfPk5N3z3LPTg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-v08U42TZqPfPk5N3z3LPTg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 19, 2018 6:04:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-v08U42TZqPfPk5N3z3LPTg.jar
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 28 files newly uploaded
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 19, 2018 6:04:14 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 19, 2018 6:04:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88177 bytes, hash nOlxKKYjQgb8RVUk8OFl2A> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-nOlxKKYjQgb8RVUk8OFl2A.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-18_22_04_15-1171071426444180883
Nov 19, 2018 6:04:16 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-18_22_04_15-1171071426444180883?project=apache-beam-testing
Nov 19, 2018 6:04:16 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-18_22_04_15-1171071426444180883
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-19T06:04:15.553Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:18.660Z: Checking permissions granted to controller Service Account.
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:25.808Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:26.420Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:26.496Z: Expanding GroupByKey operations into optimizable parts.
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:26.532Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:26.666Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:26.701Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:26.737Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:26.777Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:26.810Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:26.846Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:26.885Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:26.923Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:26.963Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:26.998Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.040Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.069Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.104Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.143Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.181Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.223Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.260Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.295Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.340Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.368Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.404Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.442Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.478Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.515Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.559Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.591Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.621Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.658Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.698Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.734Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.773Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.808Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.846Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:27.886Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:28.341Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:28.375Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:28.410Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:28.425Z: Starting 4 workers in us-central1-b...
Nov 19, 2018 6:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:28.708Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 19, 2018 6:04:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:04:47.661Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 19, 2018 6:05:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:05:16.569Z: Workers have started successfully.
Nov 19, 2018 6:05:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:05:23.284Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 19, 2018 6:05:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:05:48.186Z: Workers have started successfully.
Nov 19, 2018 6:06:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:06:34.002Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 19, 2018 6:06:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:06:34.102Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 19, 2018 6:06:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:06:35.856Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 19, 2018 6:06:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:06:35.965Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 19, 2018 6:06:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:06:37.650Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 19, 2018 6:06:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:06:37.765Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 19, 2018 6:06:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:06:41.264Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 19, 2018 6:06:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:06:41.461Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 19, 2018 6:07:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:07:07.281Z: Cleaning up.
Nov 19, 2018 6:07:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T06:07:07.446Z: Stopping worker pool...
Nov 19, 2018 6:08:16 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-19T06:08:16.403Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-19T06:08:16.444Z RUNNING Query7
2018-11-19T06:08:17.444Z new perf {"snapshots":null,"runtimeSec":95.744,"numEvents":10000000,"eventsPerSec":104445.1871657754,"eventBytesPerSec":1.3406217590658423E7,"numResults":100,"resultsPerSec":1.044451871657754,"resultBytesPerSec":105.52097259358288,"startupDelaySec":-165.31,"processingDelaySec":72.22,"shutdownDelaySec":69.566,"timeDilation":10.271139705882353,"errors":null,"jobId":null}
2018-11-19T06:08:32.534Z RUNNING Query7
2018-11-19T06:08:33.551Z no activity
2018-11-19T06:08:48.590Z RUNNING Query7
2018-11-19T06:08:49.579Z no activity
2018-11-19T06:09:04.625Z RUNNING Query7
2018-11-19T06:09:05.717Z no activity
2018-11-19T06:09:20.772Z RUNNING Query7
2018-11-19T06:09:21.797Z no activity
2018-11-19T06:09:36.847Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 4,5,main]) completed. Took 5 mins 47.517 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 25s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/5265vrvxkakuy
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1076
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1076/display/redirect>
------------------------------------------
[...truncated 404.47 KB...]
Nov 19, 2018 12:04:10 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-rHivmErBfEwxK0X3kpNi0A.jar
Nov 19, 2018 12:04:10 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 28 files newly uploaded
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.ReadBounded as step s1
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource) as step s2
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/MapElements/Map as step s3
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Monitor as step s4
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Snoop as step s5
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) as step s6
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s7
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) as step s8
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize as step s9
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) as step s10
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys as step s11
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) as step s12
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections as step s13
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView as step s14
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) as step s15
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/AsBid as step s16
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles as step s17
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Debug as step s18
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Stamp as step s19
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.Format as step s20
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.DevNull as step s21
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 19, 2018 12:04:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <58907 bytes, hash STrIFbq-2Ih5Xpgr_ndJwA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-STrIFbq-2Ih5Xpgr_ndJwA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 19, 2018 12:04:13 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-18_16_04_12-3040468906700270689
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-18_16_04_12-3040468906700270689?project=apache-beam-testing
Nov 19, 2018 12:04:13 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-18_16_04_12-3040468906700270689
Nov 19, 2018 12:04:13 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 28 files newly uploaded
Nov 19, 2018 12:04:13 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 19, 2018 12:04:13 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 19, 2018 12:04:13 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 19, 2018 12:04:13 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 19, 2018 12:04:13 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 19, 2018 12:04:13 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 19, 2018 12:04:13 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 19, 2018 12:04:13 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 19, 2018 12:04:13 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 19, 2018 12:04:13 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 19, 2018 12:04:13 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 19, 2018 12:04:13 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 19, 2018 12:04:13 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 19, 2018 12:04:13 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 19, 2018 12:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 19, 2018 12:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 19, 2018 12:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 19, 2018 12:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 19, 2018 12:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 19, 2018 12:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 19, 2018 12:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 19, 2018 12:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 19, 2018 12:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 19, 2018 12:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 19, 2018 12:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 19, 2018 12:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 19, 2018 12:04:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 19, 2018 12:04:14 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 19, 2018 12:04:14 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88177 bytes, hash C0YJRX-W2tKpcN6jGrRkKw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-C0YJRX-W2tKpcN6jGrRkKw.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-19T00:04:12.082Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:14.979Z: Checking permissions granted to controller Service Account.
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:21.889Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:22.407Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:22.603Z: Expanding GroupByKey operations into optimizable parts.
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:22.651Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:22.748Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:22.788Z: Fusing consumer JoinToFiles/JoinToFiles.Monitor into JoinToFiles.ReadBounded
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:22.829Z: Fusing consumer JoinToFiles/JoinToFiles.Snoop into JoinToFiles/JoinToFiles.Monitor
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:22.877Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) into JoinToFiles/JoinToFiles.Snoop
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:22.925Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/AsBid into JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:22.974Z: Fusing consumer JoinToFiles.Format into JoinToFiles/JoinToFiles.Stamp
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:23.022Z: Fusing consumer JoinToFiles.DevNull into JoinToFiles.Format
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:23.074Z: Fusing consumer JoinToFiles/JoinToFiles.Stamp into JoinToFiles/JoinToFiles.Debug
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:23.118Z: Fusing consumer JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles into JoinToFiles/JoinToFiles/justBids/AsBid
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:23.161Z: Fusing consumer JoinToFiles/JoinToFiles.Debug into JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:23.206Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:23.251Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:23.295Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:23.343Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) into NexmarkUtils.GenerateSideInputData/MapElements/Map
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:23.379Z: Fusing consumer NexmarkUtils.GenerateSideInputData/MapElements/Map into NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:23.425Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:23.472Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:23.508Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:24.018Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Create
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:24.065Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:24.124Z: Starting 4 workers in us-central1-b...
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:24.124Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Create
Nov 19, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:24.393Z: Executing operation NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)+NexmarkUtils.GenerateSideInputData/MapElements/Map+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 19, 2018 12:04:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:04:34.165Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 19, 2018 12:05:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:05:02.295Z: Workers have started successfully.
Nov 19, 2018 12:05:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:05:02.800Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 19, 2018 12:05:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:05:02.845Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 19, 2018 12:05:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:05:08.319Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 19, 2018 12:05:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:05:20.060Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 19, 2018 12:05:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:05:20.152Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write
Nov 19, 2018 12:05:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:05:27.419Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Close
Nov 19, 2018 12:05:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:05:27.469Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Close
Nov 19, 2018 12:05:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:05:27.517Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey)
Nov 19, 2018 12:05:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:05:27.567Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize)
Nov 19, 2018 12:05:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:05:33.462Z: Workers have started successfully.
Nov 19, 2018 12:05:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:05:34.462Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections
Nov 19, 2018 12:05:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:05:34.695Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView
Nov 19, 2018 12:05:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:05:34.923Z: Executing operation JoinToFiles.ReadBounded+JoinToFiles/JoinToFiles.Monitor+JoinToFiles/JoinToFiles.Snoop+JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)+JoinToFiles/JoinToFiles/justBids/AsBid+JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles+JoinToFiles/JoinToFiles.Debug+JoinToFiles/JoinToFiles.Stamp+JoinToFiles.Format+JoinToFiles.DevNull
Nov 19, 2018 12:06:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:06:35.112Z: Cleaning up.
Nov 19, 2018 12:06:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-19T00:06:35.232Z: Stopping worker pool...
Nov 19, 2018 12:08:13 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-19T00:08:13.198Z Waiting for main pipeline to 'finish'WARNING: No terminal state was returned. State value RUNNING
2018-11-19T00:08:13.237Z RUNNING JoinToFiles
2018-11-19T00:08:14.247Z new perf {"snapshots":null,"runtimeSec":58.648,"numEvents":10000000,"eventsPerSec":170508.7982539899,"eventBytesPerSec":2.1885910806847632E7,"numResults":9200000,"resultsPerSec":156868.0943936707,"resultBytesPerSec":5591525.627472377,"startupDelaySec":-157.042,"processingDelaySec":0.717,"shutdownDelaySec":98.394,"timeDilation":17.050027281407722,"errors":null,"jobId":null}
2018-11-19T00:08:29.310Z RUNNING JoinToFiles
2018-11-19T00:08:30.535Z no activity
2018-11-19T00:08:45.575Z RUNNING JoinToFiles
2018-11-19T00:08:46.508Z no activity
2018-11-19T00:09:01.541Z RUNNING JoinToFiles
2018-11-19T00:09:02.674Z no activity
2018-11-19T00:09:17.722Z DONE JoinToFiles
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 8,5,main]) completed. Took 5 mins 33.989 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 11s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/3ma6ubw6m4pwg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1075
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1075/display/redirect>
------------------------------------------
[...truncated 249.11 KB...]
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 18, 2018 6:04:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-hOLEWj9_NGsdxCeuIKY59g.jar
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 18, 2018 6:04:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88168 bytes, hash o2eBQXctceT-SL9Q8UtbGw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-o2eBQXctceT-SL9Q8UtbGw.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-18_10_04_07-61524305918729972
Nov 18, 2018 6:04:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-18_10_04_07-61524305918729972?project=apache-beam-testing
Nov 18, 2018 6:04:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-18_10_04_07-61524305918729972
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-18T18:04:07.426Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:10.754Z: Checking permissions granted to controller Service Account.
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:17.161Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:17.896Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:17.988Z: Expanding GroupByKey operations into optimizable parts.
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.034Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.229Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.264Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#5e4921c369c80b03
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.308Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.342Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.381Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.412Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.452Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.492Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.519Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.567Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.612Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.663Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.700Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.743Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.792Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.838Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.888Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.926Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:18.965Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:19.007Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:19.057Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:19.094Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:19.130Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:19.176Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:19.222Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:19.271Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:19.314Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:19.348Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:19.394Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:19.443Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:19.480Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:19.531Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:19.576Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:19.625Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:20.116Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:20.156Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:20.203Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:20.216Z: Starting 4 workers in us-central1-b...
Nov 18, 2018 6:04:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:20.533Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 18, 2018 6:04:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:04:37.542Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 18, 2018 6:05:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:05:05.973Z: Workers have started successfully.
Nov 18, 2018 6:05:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:05:12.312Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 18, 2018 6:05:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:05:12.363Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 18, 2018 6:05:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:05:17.870Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 18, 2018 6:05:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:05:17.911Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 18, 2018 6:05:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:05:23.486Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 18, 2018 6:05:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:05:36.182Z: Workers have started successfully.
Nov 18, 2018 6:06:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:06:27.228Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 18, 2018 6:06:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:06:27.320Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 18, 2018 6:06:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:06:29.032Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 18, 2018 6:06:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:06:29.124Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 18, 2018 6:06:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:06:30.602Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 18, 2018 6:06:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:06:30.698Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 18, 2018 6:06:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:06:34.550Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 18, 2018 6:06:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:06:34.802Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 18, 2018 6:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:07:03.036Z: Cleaning up.
Nov 18, 2018 6:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T18:07:03.223Z: Stopping worker pool...
Nov 18, 2018 6:08:08 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-18T18:08:08.734Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-18T18:08:08.774Z RUNNING Query7
2018-11-18T18:08:10.338Z new perf {"snapshots":null,"runtimeSec":103.697,"numEvents":10000000,"eventsPerSec":96434.8052499108,"eventBytesPerSec":1.237803308678168E7,"numResults":100,"resultsPerSec":0.9643480524991079,"resultBytesPerSec":97.42808374398487,"startupDelaySec":-169.826,"processingDelaySec":77.83,"shutdownDelaySec":66.129,"timeDilation":9.483398748276228,"errors":null,"jobId":null}
2018-11-18T18:08:25.384Z RUNNING Query7
2018-11-18T18:08:26.777Z no activity
2018-11-18T18:08:41.873Z RUNNING Query7
2018-11-18T18:08:43.353Z no activity
2018-11-18T18:08:58.390Z RUNNING Query7
2018-11-18T18:08:59.966Z no activity
2018-11-18T18:09:15.061Z RUNNING Query7
2018-11-18T18:09:16.633Z no activity
2018-11-18T18:09:31.672Z RUNNING Query7
2018-11-18T18:09:33.435Z no activity
2018-11-18T18:09:48.536Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 11,5,main]) completed. Took 6 mins 7.234 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 42s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/23echjarp2j7m
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1074
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1074/display/redirect>
------------------------------------------
[...truncated 265.73 KB...]
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Nov 18, 2018 12:03:46 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar>
java.io.IOException: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-npVr1jgojOCPabH0TiMmrQ.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-npVr1jgojOCPabH0TiMmrQ.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:432)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:287)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-npVr1jgojOCPabH0TiMmrQ.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-npVr1jgojOCPabH0TiMmrQ.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 18, 2018 12:03:51 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-npVr1jgojOCPabH0TiMmrQ.jar
Nov 18, 2018 12:03:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-npVr1jgojOCPabH0TiMmrQ.jar
Nov 18, 2018 12:03:55 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 28 files newly uploaded
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.ReadBounded as step s1
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource) as step s2
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/MapElements/Map as step s3
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Monitor as step s4
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Snoop as step s5
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) as step s6
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s7
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) as step s8
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize as step s9
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) as step s10
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys as step s11
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) as step s12
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections as step s13
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView as step s14
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) as step s15
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/AsBid as step s16
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles as step s17
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Debug as step s18
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Stamp as step s19
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.Format as step s20
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.DevNull as step s21
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 18, 2018 12:03:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <58914 bytes, hash LzEChJGpEJPAziB1dmB6dA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-LzEChJGpEJPAziB1dmB6dA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-18_04_03_56-5138847349962229062
Nov 18, 2018 12:03:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-18_04_03_56-5138847349962229062?project=apache-beam-testing
Nov 18, 2018 12:03:57 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-18_04_03_56-5138847349962229062
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-18T12:03:56.870Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:03:59.951Z: Checking permissions granted to controller Service Account.
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:06.637Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:07.106Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:07.281Z: Expanding GroupByKey operations into optimizable parts.
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:07.329Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:07.468Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:07.509Z: Fusing consumer JoinToFiles/JoinToFiles.Monitor into JoinToFiles.ReadBounded
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:07.554Z: Fusing consumer JoinToFiles/JoinToFiles.Snoop into JoinToFiles/JoinToFiles.Monitor
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:07.599Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) into JoinToFiles/JoinToFiles.Snoop
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:07.640Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/AsBid into JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:07.689Z: Fusing consumer JoinToFiles.Format into JoinToFiles/JoinToFiles.Stamp
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:07.759Z: Fusing consumer JoinToFiles.DevNull into JoinToFiles.Format
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:07.799Z: Fusing consumer JoinToFiles/JoinToFiles.Stamp into JoinToFiles/JoinToFiles.Debug
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:07.832Z: Fusing consumer JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles into JoinToFiles/JoinToFiles/justBids/AsBid
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:07.877Z: Fusing consumer JoinToFiles/JoinToFiles.Debug into JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:07.922Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:07.947Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:07.997Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:08.037Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) into NexmarkUtils.GenerateSideInputData/MapElements/Map
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:08.082Z: Fusing consumer NexmarkUtils.GenerateSideInputData/MapElements/Map into NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:08.117Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:08.164Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:08.210Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:08.673Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Create
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:08.709Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:08.755Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Create
Nov 18, 2018 12:04:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:08.780Z: Starting 4 workers in us-central1-b...
Nov 18, 2018 12:04:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:09.014Z: Executing operation NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)+NexmarkUtils.GenerateSideInputData/MapElements/Map+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 18, 2018 12:04:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:04:31.479Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 18, 2018 12:05:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:05:00.616Z: Workers have started successfully.
Nov 18, 2018 12:05:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:05:00.898Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 18, 2018 12:05:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:05:00.937Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 18, 2018 12:05:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:05:01.059Z: Workers have started successfully.
Nov 18, 2018 12:05:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:05:12.478Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 18, 2018 12:05:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:05:12.516Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 18, 2018 12:05:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:05:18.664Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 18, 2018 12:05:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:05:22.751Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 18, 2018 12:05:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:05:22.840Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write
Nov 18, 2018 12:05:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:05:30.041Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Close
Nov 18, 2018 12:05:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:05:30.089Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Close
Nov 18, 2018 12:05:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:05:30.153Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey)
Nov 18, 2018 12:05:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:05:30.194Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize)
Nov 18, 2018 12:05:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:05:38.512Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections
Nov 18, 2018 12:05:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:05:38.748Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView
Nov 18, 2018 12:05:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:05:38.984Z: Executing operation JoinToFiles.ReadBounded+JoinToFiles/JoinToFiles.Monitor+JoinToFiles/JoinToFiles.Snoop+JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)+JoinToFiles/JoinToFiles/justBids/AsBid+JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles+JoinToFiles/JoinToFiles.Debug+JoinToFiles/JoinToFiles.Stamp+JoinToFiles.Format+JoinToFiles.DevNull
Nov 18, 2018 12:06:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:06:37.496Z: Cleaning up.
Nov 18, 2018 12:06:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T12:06:37.595Z: Stopping worker pool...
Nov 18, 2018 12:07:58 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-18T12:07:58.050Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-18T12:07:58.086Z RUNNING JoinToFiles
2018-11-18T12:07:59.042Z new perf {"snapshots":null,"runtimeSec":56.761,"numEvents":10000000,"eventsPerSec":176177.30483959056,"eventBytesPerSec":2.2613500414016664E7,"numResults":9200000,"resultsPerSec":162083.12045242332,"resultBytesPerSec":5777413.981430911,"startupDelaySec":-137.918,"processingDelaySec":0.736,"shutdownDelaySec":81.157,"timeDilation":17.616849597434857,"errors":null,"jobId":null}
2018-11-18T12:08:14.087Z RUNNING JoinToFiles
2018-11-18T12:08:15.008Z no activity
2018-11-18T12:08:30.044Z RUNNING JoinToFiles
2018-11-18T12:08:30.920Z no activity
2018-11-18T12:08:45.959Z RUNNING JoinToFiles
2018-11-18T12:08:46.824Z no activity
2018-11-18T12:09:01.862Z DONE JoinToFiles
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 6,5,main]) completed. Took 5 mins 32.12 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 8m 54s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/g7p4mpfkrcbha
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1073
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1073/display/redirect>
------------------------------------------
[...truncated 246.56 KB...]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-fQwy7GdBX2g-kcCwqmZV4g.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-fQwy7GdBX2g-kcCwqmZV4g.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 18, 2018 6:03:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-fQwy7GdBX2g-kcCwqmZV4g.jar
Nov 18, 2018 6:04:00 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 18, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 18, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 18, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 18, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 18, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 18, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 18, 2018 6:04:01 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88159 bytes, hash IA3KSrJCz76K6VYzQH_cbg> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-IA3KSrJCz76K6VYzQH_cbg.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 18, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-17_22_04_01-18122786538372324040
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-17_22_04_01-18122786538372324040?project=apache-beam-testing
Nov 18, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-17_22_04_01-18122786538372324040
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-18T06:04:02.015Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:05.124Z: Checking permissions granted to controller Service Account.
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:11.995Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:12.769Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:13.087Z: Expanding GroupByKey operations into optimizable parts.
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:13.244Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:13.885Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:14.063Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:14.187Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:14.343Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:14.462Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:14.597Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:14.772Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:14.928Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:15.074Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:15.236Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:15.360Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:15.481Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:15.675Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:15.858Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:15.972Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:16.094Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:16.230Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:16.375Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:16.563Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:16.689Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:16.833Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:16.968Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:17.074Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:17.243Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:17.374Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:17.482Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:17.663Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:17.803Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:17.954Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:18.081Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:18.235Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:18.367Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:18.489Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:18.634Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:20.355Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:20.570Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:20.765Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 18, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:20.777Z: Starting 4 workers in us-central1-b...
Nov 18, 2018 6:04:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:21.855Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 18, 2018 6:04:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:04:38.068Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 18, 2018 6:05:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:05:05.939Z: Workers have started successfully.
Nov 18, 2018 6:05:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:05:17.965Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 18, 2018 6:05:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:05:37.157Z: Workers have started successfully.
Nov 18, 2018 6:06:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:06:26.399Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 18, 2018 6:06:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:06:26.484Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 18, 2018 6:06:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:06:29.196Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 18, 2018 6:06:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:06:29.302Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 18, 2018 6:06:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:06:30.793Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 18, 2018 6:06:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:06:30.877Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 18, 2018 6:06:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:06:34.073Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 18, 2018 6:06:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:06:34.281Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 18, 2018 6:06:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:06:57.720Z: Cleaning up.
Nov 18, 2018 6:06:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T06:06:57.858Z: Stopping worker pool...
Nov 18, 2018 6:08:03 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-18T06:08:03.071Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-18T06:08:03.101Z RUNNING Query7
2018-11-18T06:08:04.465Z new perf {"snapshots":null,"runtimeSec":92.28,"numEvents":10000000,"eventsPerSec":108365.8430862592,"eventBytesPerSec":1.3909459221933246E7,"numResults":100,"resultsPerSec":1.083658430862592,"resultBytesPerSec":109.48201127004768,"startupDelaySec":-157.972,"processingDelaySec":70.76,"shutdownDelaySec":65.692,"timeDilation":10.65669700910273,"errors":null,"jobId":null}
2018-11-18T06:08:19.511Z RUNNING Query7
2018-11-18T06:08:20.602Z no activity
2018-11-18T06:08:35.752Z RUNNING Query7
2018-11-18T06:08:36.978Z no activity
2018-11-18T06:08:52.024Z RUNNING Query7
2018-11-18T06:08:53.210Z no activity
2018-11-18T06:09:08.269Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 5,5,main]) completed. Took 5 mins 34.511 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 1s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/kffueqghcru5w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1072
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1072/display/redirect>
------------------------------------------
[...truncated 273.62 KB...]
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-0sxwgGaw3fYXWCphtvC51Q.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-0sxwgGaw3fYXWCphtvC51Q.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 18, 2018 12:04:05 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-0sxwgGaw3fYXWCphtvC51Q.jar
Nov 18, 2018 12:04:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-0sxwgGaw3fYXWCphtvC51Q.jar
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 18, 2018 12:04:15 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88174 bytes, hash v5JJSZendhWnerfMjxylqw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-v5JJSZendhWnerfMjxylqw.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 18, 2018 12:04:17 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-17_16_04_16-13198916419471589042
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-17_16_04_16-13198916419471589042?project=apache-beam-testing
Nov 18, 2018 12:04:17 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-17_16_04_16-13198916419471589042
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-18T00:04:16.379Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:19.847Z: Checking permissions granted to controller Service Account.
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:27.325Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:27.808Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:27.917Z: Expanding GroupByKey operations into optimizable parts.
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:27.981Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:28.215Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:28.273Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#e9b79e6542892c55
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:28.324Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:28.369Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:28.414Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:28.461Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:28.511Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:28.557Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:28.607Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:28.667Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:28.714Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:28.774Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:28.844Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:28.892Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:28.956Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:29.003Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:29.052Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:29.100Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:29.146Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:29.403Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:29.456Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:29.501Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:29.551Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:29.589Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:29.652Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:29.701Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:29.766Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:29.812Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:29.858Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:29.908Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:29.979Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:30.027Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:30.073Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:30.124Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:30.842Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:30.920Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:31.005Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 18, 2018 12:04:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:31.006Z: Starting 4 workers in us-central1-b...
Nov 18, 2018 12:04:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:31.400Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 18, 2018 12:04:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:04:43.378Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 18, 2018 12:05:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:05:14.195Z: Workers have started successfully.
Nov 18, 2018 12:05:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:05:18.650Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 18, 2018 12:05:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:05:18.709Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 18, 2018 12:05:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:05:24.895Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 18, 2018 12:05:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:05:45.752Z: Workers have started successfully.
Nov 18, 2018 12:06:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:06:27.712Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 18, 2018 12:06:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:06:27.884Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 18, 2018 12:06:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:06:29.497Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 18, 2018 12:06:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:06:29.627Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 18, 2018 12:06:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:06:31.847Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 18, 2018 12:06:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:06:32.112Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 18, 2018 12:06:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:06:35.219Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 18, 2018 12:06:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:06:35.460Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 18, 2018 12:07:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:07:00.989Z: Cleaning up.
Nov 18, 2018 12:07:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-18T00:07:01.152Z: Stopping worker pool...
Nov 18, 2018 12:08:17 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-18T00:08:17.415Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-18T00:08:17.456Z RUNNING Query7
2018-11-18T00:08:18.588Z new perf {"snapshots":null,"runtimeSec":92.088,"numEvents":10000000,"eventsPerSec":108591.78177395536,"eventBytesPerSec":1.3938459918773348E7,"numResults":100,"resultsPerSec":1.0859178177395536,"resultBytesPerSec":109.7102771262271,"startupDelaySec":-168.981,"processingDelaySec":69.571,"shutdownDelaySec":76.893,"timeDilation":10.678915819650769,"errors":null,"jobId":null}
2018-11-18T00:08:33.638Z RUNNING Query7
2018-11-18T00:08:34.630Z no activity
2018-11-18T00:08:49.669Z RUNNING Query7
2018-11-18T00:08:50.690Z no activity
2018-11-18T00:09:05.747Z RUNNING Query7
2018-11-18T00:09:06.780Z no activity
2018-11-18T00:09:21.819Z RUNNING Query7
2018-11-18T00:09:22.852Z no activity
2018-11-18T00:09:37.914Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 9,5,main]) completed. Took 5 mins 51.708 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 31s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/go43wfzywac6s
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1071
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1071/display/redirect?page=changes>
Changes:
[gleb] [BEAM-5866] Fix `Row#equals`
------------------------------------------
[...truncated 252.27 KB...]
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-FgNfz2eV-iN6aWbLHtiP0A.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-FgNfz2eV-iN6aWbLHtiP0A.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Nov 17, 2018 7:51:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar>
java.io.IOException: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-FgNfz2eV-iN6aWbLHtiP0A.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-FgNfz2eV-iN6aWbLHtiP0A.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:432)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:287)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-FgNfz2eV-iN6aWbLHtiP0A.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-FgNfz2eV-iN6aWbLHtiP0A.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 17, 2018 7:51:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-FgNfz2eV-iN6aWbLHtiP0A.jar
Nov 17, 2018 7:52:01 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.ReadBounded as step s1
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource) as step s2
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/MapElements/Map as step s3
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Monitor as step s4
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Snoop as step s5
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) as step s6
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s7
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) as step s8
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize as step s9
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) as step s10
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys as step s11
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) as step s12
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections as step s13
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView as step s14
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) as step s15
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/AsBid as step s16
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles as step s17
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Debug as step s18
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Stamp as step s19
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.Format as step s20
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.DevNull as step s21
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 17, 2018 7:52:02 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <58917 bytes, hash ZgNwwK7r1hvWltTNysKD5Q> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-ZgNwwK7r1hvWltTNysKD5Q.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-17_11_52_02-10313966891373507851
Nov 17, 2018 7:52:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-17_11_52_02-10313966891373507851?project=apache-beam-testing
Nov 17, 2018 7:52:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-17_11_52_02-10313966891373507851
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-17T19:52:02.708Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:05.462Z: Checking permissions granted to controller Service Account.
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:11.788Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:12.274Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:12.419Z: Expanding GroupByKey operations into optimizable parts.
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:12.444Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:12.566Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:12.604Z: Fusing consumer JoinToFiles/JoinToFiles.Monitor into JoinToFiles.ReadBounded
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:12.630Z: Fusing consumer JoinToFiles/JoinToFiles.Snoop into JoinToFiles/JoinToFiles.Monitor
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:12.669Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) into JoinToFiles/JoinToFiles.Snoop
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:12.705Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/AsBid into JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:12.743Z: Fusing consumer JoinToFiles.Format into JoinToFiles/JoinToFiles.Stamp
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:12.775Z: Fusing consumer JoinToFiles.DevNull into JoinToFiles.Format
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:12.812Z: Fusing consumer JoinToFiles/JoinToFiles.Stamp into JoinToFiles/JoinToFiles.Debug
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:12.848Z: Fusing consumer JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles into JoinToFiles/JoinToFiles/justBids/AsBid
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:12.885Z: Fusing consumer JoinToFiles/JoinToFiles.Debug into JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:12.920Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:12.956Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:12.993Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:13.029Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) into NexmarkUtils.GenerateSideInputData/MapElements/Map
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:13.055Z: Fusing consumer NexmarkUtils.GenerateSideInputData/MapElements/Map into NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:13.087Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:13.119Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:13.154Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:13.506Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Create
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:13.538Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:13.571Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Create
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:13.587Z: Starting 4 workers in us-central1-b...
Nov 17, 2018 7:52:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:13.786Z: Executing operation NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)+NexmarkUtils.GenerateSideInputData/MapElements/Map+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 17, 2018 7:52:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:23.948Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 17, 2018 7:52:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:49.503Z: Workers have started successfully.
Nov 17, 2018 7:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:53.219Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 17, 2018 7:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:53.258Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 17, 2018 7:52:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:53.567Z: Workers have started successfully.
Nov 17, 2018 7:53:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:52:59.068Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 17, 2018 7:53:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:53:11.001Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 17, 2018 7:53:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:53:11.077Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write
Nov 17, 2018 7:53:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:53:18.117Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Close
Nov 17, 2018 7:53:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:53:18.156Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Close
Nov 17, 2018 7:53:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:53:18.207Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey)
Nov 17, 2018 7:53:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:53:18.272Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize)
Nov 17, 2018 7:53:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:53:26.713Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections
Nov 17, 2018 7:53:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:53:26.919Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView
Nov 17, 2018 7:53:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:53:27.094Z: Executing operation JoinToFiles.ReadBounded+JoinToFiles/JoinToFiles.Monitor+JoinToFiles/JoinToFiles.Snoop+JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)+JoinToFiles/JoinToFiles/justBids/AsBid+JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles+JoinToFiles/JoinToFiles.Debug+JoinToFiles/JoinToFiles.Stamp+JoinToFiles.Format+JoinToFiles.DevNull
Nov 17, 2018 7:54:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:54:28.768Z: Cleaning up.
Nov 17, 2018 7:54:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T19:54:28.924Z: Stopping worker pool...
Nov 17, 2018 7:56:03 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-11-17T19:56:03.794Z Waiting for main pipeline to 'finish'
2018-11-17T19:56:03.824Z RUNNING JoinToFiles
2018-11-17T19:56:04.887Z new perf {"snapshots":null,"runtimeSec":59.19,"numEvents":10000000,"eventsPerSec":168947.45734076703,"eventBytesPerSec":2.1685502568001352E7,"numResults":9200000,"resultsPerSec":155431.66075350565,"resultBytesPerSec":5540324.294644366,"startupDelaySec":-154.429,"processingDelaySec":0.828,"shutdownDelaySec":95.239,"timeDilation":16.89390099679,"errors":null,"jobId":null}
2018-11-17T19:56:19.945Z RUNNING JoinToFiles
2018-11-17T19:56:20.864Z no activity
2018-11-17T19:56:35.907Z RUNNING JoinToFiles
2018-11-17T19:56:36.891Z no activity
2018-11-17T19:56:51.933Z RUNNING JoinToFiles
2018-11-17T19:56:52.964Z no activity
2018-11-17T19:57:08.027Z DONE JoinToFiles
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 6,5,main]) completed. Took 5 mins 32.445 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 23s
76 actionable tasks: 72 executed, 4 from cache
Publishing build scan...
https://gradle.com/s/hvjedbmkbiapa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1070
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1070/display/redirect>
------------------------------------------
[...truncated 249.29 KB...]
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 17, 2018 6:04:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Gijun_Mjb3-00Hy4AYwBJg.jar
Nov 17, 2018 6:04:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Gijun_Mjb3-00Hy4AYwBJg.jar
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 17, 2018 6:05:02 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 17, 2018 6:05:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88175 bytes, hash cnW8_G-y8EWof0i8saNJlQ> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-cnW8_G-y8EWof0i8saNJlQ.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-17_10_05_03-11346866341749024139
Nov 17, 2018 6:05:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-17_10_05_03-11346866341749024139?project=apache-beam-testing
Nov 17, 2018 6:05:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-17_10_05_03-11346866341749024139
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-17T18:05:03.467Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:06.711Z: Checking permissions granted to controller Service Account.
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:13.119Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:13.538Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:13.629Z: Expanding GroupByKey operations into optimizable parts.
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:13.664Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:13.860Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:13.903Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:13.949Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:13.997Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.044Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.091Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.139Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.189Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.231Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.278Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.320Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.364Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.407Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.451Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.499Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.546Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.590Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.624Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.669Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.714Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.762Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.807Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.847Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.881Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.914Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:14.960Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:15.006Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:15.042Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:15.087Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:15.136Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:15.184Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:15.231Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:15.272Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:15.316Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:15.765Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:15.814Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:15.860Z: Starting 4 workers in us-central1-b...
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:15.861Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 17, 2018 6:05:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:16.104Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 17, 2018 6:05:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:36.804Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 17, 2018 6:06:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:59.755Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 17, 2018 6:06:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:05:59.794Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 17, 2018 6:06:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:06:05.269Z: Workers have started successfully.
Nov 17, 2018 6:06:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:06:05.312Z: Workers have started successfully.
Nov 17, 2018 6:06:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:06:17.213Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 17, 2018 6:06:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:06:17.253Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 17, 2018 6:06:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:06:22.697Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 17, 2018 6:07:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:07:27.117Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 17, 2018 6:07:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:07:27.215Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 17, 2018 6:07:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:07:29.983Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 17, 2018 6:07:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:07:30.084Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 17, 2018 6:07:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:07:32.605Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 17, 2018 6:07:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:07:32.701Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 17, 2018 6:07:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:07:35.979Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 17, 2018 6:07:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:07:36.186Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 17, 2018 6:08:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:08:02.438Z: Cleaning up.
Nov 17, 2018 6:08:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T18:08:02.603Z: Stopping worker pool...
Nov 17, 2018 6:09:04 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-17T18:09:04.526Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-17T18:09:04.574Z RUNNING Query7
2018-11-17T18:09:05.807Z new perf {"snapshots":null,"runtimeSec":103.15,"numEvents":10000000,"eventsPerSec":96946.19486185166,"eventBytesPerSec":1.2443673262239456E7,"numResults":100,"resultsPerSec":0.9694619486185166,"resultBytesPerSec":97.94474066892874,"startupDelaySec":-165.891,"processingDelaySec":79.787,"shutdownDelaySec":62.741,"timeDilation":9.533688802714492,"errors":null,"jobId":null}
2018-11-17T18:09:20.845Z RUNNING Query7
2018-11-17T18:09:22.067Z no activity
2018-11-17T18:09:37.324Z RUNNING Query7
2018-11-17T18:09:38.528Z no activity
2018-11-17T18:09:53.595Z RUNNING Query7
2018-11-17T18:09:54.800Z no activity
2018-11-17T18:10:09.852Z RUNNING Query7
2018-11-17T18:10:11.327Z no activity
2018-11-17T18:10:26.376Z RUNNING Query7
2018-11-17T18:10:27.685Z no activity
2018-11-17T18:10:42.727Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 2,5,main]) completed. Took 5 mins 59.094 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 36s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/rrfa3htrbbk6g
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1069
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1069/display/redirect>
------------------------------------------
[...truncated 241.17 KB...]
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-EO2v1OooyrCHVe_Hq5b2KQ.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-EO2v1OooyrCHVe_Hq5b2KQ.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 17, 2018 12:03:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-EO2v1OooyrCHVe_Hq5b2KQ.jar
Nov 17, 2018 12:03:51 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 117 files cached, 28 files newly uploaded
Nov 17, 2018 12:03:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 17, 2018 12:03:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 17, 2018 12:03:52 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88179 bytes, hash REcoXUgBiXhG79yclr0fxg> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-REcoXUgBiXhG79yclr0fxg.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 17, 2018 12:03:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-17_04_03_52-16600435183677834497
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-17_04_03_52-16600435183677834497?project=apache-beam-testing
Nov 17, 2018 12:03:53 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-17_04_03_52-16600435183677834497
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-17T12:03:52.676Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:03:55.784Z: Checking permissions granted to controller Service Account.
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:02.363Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:03.211Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:03.317Z: Expanding GroupByKey operations into optimizable parts.
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:03.364Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:03.568Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:03.611Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:03.648Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:03.695Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:03.742Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:03.788Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:03.835Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:03.882Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:03.927Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:03.975Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.021Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.066Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.105Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.148Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.193Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.234Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.281Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.327Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.373Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.419Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.465Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.502Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.547Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.583Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.629Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.676Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.721Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.760Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.798Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.833Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.880Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.922Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:04.969Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:05.006Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:05.490Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:05.537Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:05.574Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:05.585Z: Starting 4 workers in us-central1-b...
Nov 17, 2018 12:04:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:05.888Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 17, 2018 12:04:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:24.101Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 17, 2018 12:04:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:49.193Z: Workers have started successfully.
Nov 17, 2018 12:04:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:53.331Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 17, 2018 12:04:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:53.364Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 17, 2018 12:04:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:57.434Z: Workers have started successfully.
Nov 17, 2018 12:04:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:04:59.064Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 17, 2018 12:06:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:06:05.834Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 17, 2018 12:06:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:06:05.941Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 17, 2018 12:06:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:06:07.596Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 17, 2018 12:06:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:06:07.680Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 17, 2018 12:06:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:06:08.149Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 17, 2018 12:06:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:06:08.263Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 17, 2018 12:06:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:06:11.874Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 17, 2018 12:06:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:06:12.075Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 17, 2018 12:06:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:06:37.168Z: Cleaning up.
Nov 17, 2018 12:06:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T12:06:37.277Z: Stopping worker pool...
Nov 17, 2018 12:07:53 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-17T12:07:53.497Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-17T12:07:53.534Z RUNNING Query7
2018-11-17T12:07:54.669Z new perf {"snapshots":null,"runtimeSec":92.967,"numEvents":10000000,"eventsPerSec":107565.0499639657,"eventBytesPerSec":1.380667222777975E7,"numResults":100,"resultsPerSec":1.0756504996396572,"resultBytesPerSec":108.67296997859455,"startupDelaySec":-170.674,"processingDelaySec":70.958,"shutdownDelaySec":77.707,"timeDilation":10.577947013456388,"errors":null,"jobId":null}
2018-11-17T12:08:09.707Z RUNNING Query7
2018-11-17T12:08:10.749Z no activity
2018-11-17T12:08:25.780Z RUNNING Query7
2018-11-17T12:08:26.699Z no activity
2018-11-17T12:08:41.732Z RUNNING Query7
2018-11-17T12:08:42.777Z no activity
2018-11-17T12:08:57.819Z RUNNING Query7
2018-11-17T12:08:58.776Z no activity
2018-11-17T12:09:13.842Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 5,5,main]) completed. Took 5 mins 40.338 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 7s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/h7mejxpzuliyw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1068
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1068/display/redirect?page=changes>
Changes:
[ankurgoenka] Update dataflow container version to beam-master-20181116
------------------------------------------
[...truncated 266.59 KB...]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-zEIGYVtdaKz0g4LZuf-6uw.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-zEIGYVtdaKz0g4LZuf-6uw.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 17, 2018 6:31:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-zEIGYVtdaKz0g4LZuf-6uw.jar
Nov 17, 2018 6:31:59 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 17, 2018 6:32:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88187 bytes, hash mDIyWdfFqy4yFAhL4eVhOg> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-mDIyWdfFqy4yFAhL4eVhOg.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 17, 2018 6:32:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-16_22_32_00-982867446819691785
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_22_32_00-982867446819691785?project=apache-beam-testing
Nov 17, 2018 6:32:01 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_22_32_00-982867446819691785
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-17T06:32:01.040Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:04.035Z: Checking permissions granted to controller Service Account.
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:08.769Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:09.256Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:09.327Z: Expanding GroupByKey operations into optimizable parts.
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:09.354Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:09.508Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:09.537Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#8bc528a076ecd5d7
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:09.570Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:09.602Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:09.652Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:09.683Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:09.720Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:09.756Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:09.794Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:09.831Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:09.869Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:09.908Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:09.945Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:09.978Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.016Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.050Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.088Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.123Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.160Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.197Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.232Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.272Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.302Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.341Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.379Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.415Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.452Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.501Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.531Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.566Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.604Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.645Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.707Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:10.751Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:11.147Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 17, 2018 6:32:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:11.185Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 17, 2018 6:32:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:11.219Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 17, 2018 6:32:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:11.233Z: Starting 4 workers in us-central1-b...
Nov 17, 2018 6:32:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:11.475Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 17, 2018 6:32:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:22.795Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 17, 2018 6:32:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:47.919Z: Workers have started successfully.
Nov 17, 2018 6:32:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:51.746Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 17, 2018 6:32:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:32:52.829Z: Workers have started successfully.
Nov 17, 2018 6:34:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:34:06.921Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 17, 2018 6:34:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:34:07.050Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 17, 2018 6:34:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:34:09.503Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 17, 2018 6:34:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:34:09.589Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 17, 2018 6:34:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:34:11.451Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 17, 2018 6:34:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:34:11.531Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 17, 2018 6:34:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:34:14.745Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 17, 2018 6:34:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:34:14.909Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 17, 2018 6:34:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:34:39.607Z: Cleaning up.
Nov 17, 2018 6:34:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:34:39.712Z: Stopping worker pool...
Nov 17, 2018 6:36:01 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-17T06:36:01.935Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-17T06:36:01.970Z RUNNING Query7
2018-11-17T06:36:03.066Z new perf {"snapshots":null,"runtimeSec":95.999,"numEvents":10000000,"eventsPerSec":104167.75174741405,"eventBytesPerSec":1.3370606954239108E7,"numResults":100,"resultsPerSec":1.0416775174741404,"resultBytesPerSec":105.24067959041241,"startupDelaySec":-178.678,"processingDelaySec":73.973,"shutdownDelaySec":82.679,"timeDilation":10.243856706840697,"errors":null,"jobId":null}
2018-11-17T06:36:18.103Z RUNNING Query7
2018-11-17T06:36:19.143Z no activity
2018-11-17T06:36:34.180Z RUNNING Query7
2018-11-17T06:36:35.146Z no activity
2018-11-17T06:36:50.188Z RUNNING Query7
2018-11-17T06:36:51.166Z no activity
2018-11-17T06:37:06.211Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 7,5,main]) completed. Took 5 mins 33.443 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 14s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/ruxj5qqntjw5o
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1067
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1067/display/redirect>
------------------------------------------
[...truncated 270.42 KB...]
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-xiPTwA88ZpBl4Dy-Iwz4Jg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-xiPTwA88ZpBl4Dy-Iwz4Jg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 17, 2018 6:04:03 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-xiPTwA88ZpBl4Dy-Iwz4Jg.jar
Nov 17, 2018 6:04:04 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-xiPTwA88ZpBl4Dy-Iwz4Jg.jar
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 17, 2018 6:04:07 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 17, 2018 6:04:08 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88173 bytes, hash 2SDjnlOOY_7BGU9K4Vwa5g> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-2SDjnlOOY_7BGU9K4Vwa5g.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 17, 2018 6:04:09 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-16_22_04_08-2015607738991981482
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_22_04_08-2015607738991981482?project=apache-beam-testing
Nov 17, 2018 6:04:09 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_22_04_08-2015607738991981482
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-17T06:04:08.463Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:10.896Z: Checking permissions granted to controller Service Account.
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:16.200Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:16.764Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:16.836Z: Expanding GroupByKey operations into optimizable parts.
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:16.875Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.046Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.077Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.113Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.148Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.185Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.226Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.265Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.298Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.339Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.390Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.427Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.463Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.508Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.563Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.591Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.625Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.659Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.697Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.735Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.755Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.794Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.825Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.855Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.883Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.918Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.955Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:17.993Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:18.030Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:18.068Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:18.105Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:18.140Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:18.176Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:18.214Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:18.240Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:18.634Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:18.660Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:18.687Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:18.719Z: Starting 4 workers in us-central1-b...
Nov 17, 2018 6:04:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:18.928Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 17, 2018 6:04:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:31.997Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 17, 2018 6:04:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:55.665Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 17, 2018 6:04:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:55.693Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 17, 2018 6:04:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:04:59.275Z: Workers have started successfully.
Nov 17, 2018 6:05:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:05:00.431Z: Workers have started successfully.
Nov 17, 2018 6:05:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:05:01.780Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 17, 2018 6:06:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:06:15.687Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 17, 2018 6:06:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:06:15.777Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 17, 2018 6:06:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:06:17.325Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 17, 2018 6:06:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:06:17.418Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 17, 2018 6:06:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:06:21.927Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 17, 2018 6:06:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:06:22.007Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 17, 2018 6:06:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:06:24.738Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 17, 2018 6:06:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:06:24.944Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 17, 2018 6:06:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:06:50.645Z: Cleaning up.
Nov 17, 2018 6:06:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T06:06:50.759Z: Stopping worker pool...
Nov 17, 2018 6:08:09 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-17T06:08:09.305Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-17T06:08:09.344Z RUNNING Query7
2018-11-17T06:08:10.367Z new perf {"snapshots":null,"runtimeSec":97.273,"numEvents":10000000,"eventsPerSec":102803.45008378482,"eventBytesPerSec":1.319548998180379E7,"numResults":100,"resultsPerSec":1.0280345008378482,"resultBytesPerSec":103.8623256196478,"startupDelaySec":-176.19,"processingDelaySec":73.216,"shutdownDelaySec":78.917,"timeDilation":10.109691281239398,"errors":null,"jobId":null}
2018-11-17T06:08:25.404Z RUNNING Query7
2018-11-17T06:08:26.344Z no activity
2018-11-17T06:08:41.414Z RUNNING Query7
2018-11-17T06:08:42.432Z no activity
2018-11-17T06:08:57.480Z RUNNING Query7
2018-11-17T06:08:58.440Z no activity
2018-11-17T06:09:13.481Z RUNNING Query7
2018-11-17T06:09:14.487Z no activity
2018-11-17T06:09:29.535Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':',5,main]) completed. Took 5 mins 52.126 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 23s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/hpctz4dskl4k4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1066
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1066/display/redirect?page=changes>
Changes:
[github] Remove conscrypt as security provider. (#7056)
------------------------------------------
[...truncated 257.18 KB...]
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-s7Rf9EvVsVAMSp_2uvVnbg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-s7Rf9EvVsVAMSp_2uvVnbg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 17, 2018 2:37:39 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-s7Rf9EvVsVAMSp_2uvVnbg.jar
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 17, 2018 2:37:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88181 bytes, hash hQxNtwb88nG5eozx87CAVw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-hQxNtwb88nG5eozx87CAVw.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 17, 2018 2:37:45 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-16_18_37_44-18310074388801088728
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_18_37_44-18310074388801088728?project=apache-beam-testing
Nov 17, 2018 2:37:45 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_18_37_44-18310074388801088728
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-17T02:37:44.128Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:46.778Z: Checking permissions granted to controller Service Account.
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:50.990Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:51.431Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:51.509Z: Expanding GroupByKey operations into optimizable parts.
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:51.546Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:51.704Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:51.747Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:51.789Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:51.823Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:51.850Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:51.887Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:51.925Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:51.963Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.000Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.039Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.077Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.118Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.157Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.196Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.234Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.270Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.312Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.351Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.381Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.421Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.464Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.501Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.533Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.570Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.608Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.636Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.677Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.712Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.752Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.789Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.829Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.866Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.905Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:52.944Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:53.388Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:53.422Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:53.456Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:53.475Z: Starting 4 workers in us-central1-b...
Nov 17, 2018 2:37:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:37:53.733Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 17, 2018 2:38:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:38:04.421Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 17, 2018 2:38:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:38:31.890Z: Workers have started successfully.
Nov 17, 2018 2:38:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:38:32.783Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 17, 2018 2:38:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:38:32.820Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 17, 2018 2:38:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:38:38.512Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 17, 2018 2:38:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:38:39.625Z: Workers have started successfully.
Nov 17, 2018 2:39:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:39:53.118Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 17, 2018 2:39:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:39:53.272Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 17, 2018 2:39:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:39:55.638Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 17, 2018 2:39:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:39:55.719Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 17, 2018 2:39:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:39:56.193Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 17, 2018 2:39:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:39:56.282Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 17, 2018 2:40:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:40:00.490Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 17, 2018 2:40:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:40:00.727Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 17, 2018 2:40:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:40:28.160Z: Cleaning up.
Nov 17, 2018 2:40:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:40:28.275Z: Stopping worker pool...
Nov 17, 2018 2:41:45 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-11-17T02:41:45.320Z Waiting for main pipeline to 'finish'
2018-11-17T02:41:45.488Z RUNNING Query7
2018-11-17T02:41:47.169Z new perf {"snapshots":null,"runtimeSec":101.381,"numEvents":10000000,"eventsPerSec":98637.81181878262,"eventBytesPerSec":1.2660803276748108E7,"numResults":100,"resultsPerSec":0.9863781181878262,"resultBytesPerSec":99.65378128051607,"startupDelaySec":-179.936,"processingDelaySec":77.076,"shutdownDelaySec":78.555,"timeDilation":9.700042414259082,"errors":null,"jobId":null}
2018-11-17T02:42:02.377Z RUNNING Query7
2018-11-17T02:42:03.576Z no activity
2018-11-17T02:42:18.627Z RUNNING Query7
2018-11-17T02:42:19.685Z no activity
2018-11-17T02:42:34.724Z RUNNING Query7
2018-11-17T02:42:35.762Z no activity
2018-11-17T02:42:50.805Z RUNNING Query7
2018-11-17T02:42:51.906Z no activity
2018-11-17T02:43:06.956Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 7,5,main]) completed. Took 5 mins 49.161 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 31s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/spihrtrz36las
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1065
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1065/display/redirect>
------------------------------------------
[...truncated 247.03 KB...]
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-fetSJcb_GaLoHemH87T0Ng.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 17, 2018 2:27:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-fetSJcb_GaLoHemH87T0Ng.jar
Nov 17, 2018 2:27:44 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-fetSJcb_GaLoHemH87T0Ng.jar
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 17, 2018 2:27:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88182 bytes, hash 4l2tBvPE5-8IBKNd-Igm0w> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-4l2tBvPE5-8IBKNd-Igm0w.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-16_18_27_48-17261874801735851493
Nov 17, 2018 2:27:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_18_27_48-17261874801735851493?project=apache-beam-testing
Nov 17, 2018 2:27:48 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_18_27_48-17261874801735851493
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-17T02:27:48.159Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:50.963Z: Checking permissions granted to controller Service Account.
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:55.446Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:55.836Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:55.943Z: Expanding GroupByKey operations into optimizable parts.
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:55.980Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.138Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.175Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#6044f02ac45a6f71
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.209Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.246Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.288Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.325Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.359Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.398Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.436Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.473Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.506Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.544Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.577Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.616Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.652Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.687Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.725Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.762Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.785Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.823Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.862Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.894Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.929Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:56.967Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:57.005Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:57.038Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:57.078Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:57.114Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:57.150Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:57.192Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:57.225Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:57.257Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:57.287Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:57.327Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:57.745Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:57.775Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:57.800Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:57.824Z: Starting 4 workers in us-central1-b...
Nov 17, 2018 2:27:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:27:58.020Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 17, 2018 2:28:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:28:15.294Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 17, 2018 2:28:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:28:42.014Z: Workers have started successfully.
Nov 17, 2018 2:28:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:28:44.222Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 17, 2018 2:28:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:28:44.260Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 17, 2018 2:29:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:29:02.006Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 17, 2018 2:29:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:29:12.191Z: Workers have started successfully.
Nov 17, 2018 2:30:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:30:05.102Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 17, 2018 2:30:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:30:05.197Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 17, 2018 2:30:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:30:07.797Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 17, 2018 2:30:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:30:07.873Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 17, 2018 2:30:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:30:08.336Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 17, 2018 2:30:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:30:08.420Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 17, 2018 2:30:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:30:12.435Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 17, 2018 2:30:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:30:12.629Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 17, 2018 2:30:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:30:37.517Z: Cleaning up.
Nov 17, 2018 2:30:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:30:37.611Z: Stopping worker pool...
Nov 17, 2018 2:31:49 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-11-17T02:31:49.060Z Waiting for main pipeline to 'finish'
2018-11-17T02:31:49.101Z RUNNING Query7
2018-11-17T02:31:50.330Z new perf {"snapshots":null,"runtimeSec":101.185,"numEvents":10000000,"eventsPerSec":98828.8777980926,"eventBytesPerSec":1.268532783515343E7,"numResults":100,"resultsPerSec":0.988288777980926,"resultBytesPerSec":99.84681523941295,"startupDelaySec":-173.1,"processingDelaySec":78.31,"shutdownDelaySec":71.915,"timeDilation":9.718831842664427,"errors":null,"jobId":null}
2018-11-17T02:32:05.374Z RUNNING Query7
2018-11-17T02:32:06.479Z no activity
2018-11-17T02:32:21.525Z RUNNING Query7
2018-11-17T02:32:22.583Z no activity
2018-11-17T02:32:37.618Z RUNNING Query7
2018-11-17T02:32:38.631Z no activity
2018-11-17T02:32:53.682Z RUNNING Query7
2018-11-17T02:32:54.732Z no activity
2018-11-17T02:33:09.776Z RUNNING Query7
2018-11-17T02:33:10.479Z no activity
2018-11-17T02:33:25.479Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 3,5,main]) completed. Took 5 mins 56.086 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 56s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/bsmhoytuzq5c4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1064
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1064/display/redirect?page=changes>
Changes:
[markliu] [BEAM-4850] Remove some dependencies of apitools
[robinyqiu] Redo of buggy PR #6903
------------------------------------------
[...truncated 250.79 KB...]
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-vwHXRCv8D8_MmajD_doMrA.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 17, 2018 2:17:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-vwHXRCv8D8_MmajD_doMrA.jar
Nov 17, 2018 2:17:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-vwHXRCv8D8_MmajD_doMrA.jar
Nov 17, 2018 2:17:39 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 17, 2018 2:17:40 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88175 bytes, hash r1PXMPOvWh3oR-thKywbMw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-r1PXMPOvWh3oR-thKywbMw.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 17, 2018 2:17:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-16_18_17_40-12797514707837819512
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_18_17_40-12797514707837819512?project=apache-beam-testing
Nov 17, 2018 2:17:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_18_17_40-12797514707837819512
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-17T02:17:40.796Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:43.416Z: Checking permissions granted to controller Service Account.
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:49.269Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:49.856Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:49.925Z: Expanding GroupByKey operations into optimizable parts.
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:49.951Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.114Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.153Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#ab02df29d80e6708
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.191Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.230Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.266Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.304Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.342Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.378Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.414Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.452Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.482Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.519Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.555Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.592Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.635Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.672Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.709Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.741Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.770Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.796Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.829Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.865Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.906Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.946Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:50.985Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:51.026Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:51.064Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:51.097Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:51.132Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:51.163Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:51.201Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:51.239Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:51.276Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:51.314Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:51.659Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:51.687Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:51.723Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:51.736Z: Starting 4 workers in us-central1-b...
Nov 17, 2018 2:17:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:17:51.973Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 17, 2018 2:18:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:18:02.778Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 17, 2018 2:18:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:18:43.312Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 17, 2018 2:18:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:18:43.340Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 17, 2018 2:18:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:18:44.002Z: Workers have started successfully.
Nov 17, 2018 2:18:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:18:44.048Z: Workers have started successfully.
Nov 17, 2018 2:18:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:18:49.376Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 17, 2018 2:19:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:19:56.514Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 17, 2018 2:19:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:19:56.591Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 17, 2018 2:19:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:19:58.994Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 17, 2018 2:19:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:19:59.068Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 17, 2018 2:20:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:20:00.580Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 17, 2018 2:20:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:20:00.639Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 17, 2018 2:20:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:20:04.241Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 17, 2018 2:20:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:20:04.408Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 17, 2018 2:20:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:20:30.235Z: Cleaning up.
Nov 17, 2018 2:20:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T02:20:30.374Z: Stopping worker pool...
Nov 17, 2018 2:21:41 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-17T02:21:41.762Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-17T02:21:41.797Z RUNNING Query7
2018-11-17T02:21:42.955Z new perf {"snapshots":null,"runtimeSec":92.755,"numEvents":10000000,"eventsPerSec":107810.89968195785,"eventBytesPerSec":1.3838228634574955E7,"numResults":100,"resultsPerSec":1.0781089968195785,"resultBytesPerSec":108.92135194868202,"startupDelaySec":-164.644,"processingDelaySec":69.209,"shutdownDelaySec":71.889,"timeDilation":10.602123874723734,"errors":null,"jobId":null}
2018-11-17T02:21:58.002Z RUNNING Query7
2018-11-17T02:21:59.076Z no activity
2018-11-17T02:22:14.119Z RUNNING Query7
2018-11-17T02:22:15.282Z no activity
2018-11-17T02:22:30.329Z RUNNING Query7
2018-11-17T02:22:31.886Z no activity
2018-11-17T02:22:46.926Z RUNNING Query7
2018-11-17T02:22:48.440Z no activity
2018-11-17T02:23:03.492Z RUNNING Query7
2018-11-17T02:23:04.752Z no activity
2018-11-17T02:23:19.810Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 6,5,main]) completed. Took 6 mins 2.458 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 8s
76 actionable tasks: 67 executed, 9 from cache
Publishing build scan...
https://gradle.com/s/kwtmlwasi65sw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1063
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1063/display/redirect?page=changes>
Changes:
[ankurgoenka] Revert "[BEAM-6080] Rollback PR/7049 for failing java post-commits"
[ankurgoenka] [BEAM-6080] Optionally check the sha256 for staged artifacts
------------------------------------------
[...truncated 271.26 KB...]
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 17, 2018 1:10:09 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-X8R93xgBsunEUvNDlqCPqg.jar
Nov 17, 2018 1:10:09 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-X8R93xgBsunEUvNDlqCPqg.jar
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 17, 2018 1:10:12 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 17, 2018 1:10:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88172 bytes, hash beK4lshrhy1zRV0AjA7aIQ> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-beK4lshrhy1zRV0AjA7aIQ.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 17, 2018 1:10:14 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-16_17_10_13-8264640809961244975
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_17_10_13-8264640809961244975?project=apache-beam-testing
Nov 17, 2018 1:10:14 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_17_10_13-8264640809961244975
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-17T01:10:13.443Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:16.311Z: Checking permissions granted to controller Service Account.
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:20.529Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:21.120Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:21.247Z: Expanding GroupByKey operations into optimizable parts.
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:21.297Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:21.498Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:21.550Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:21.616Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:21.654Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:21.693Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:21.745Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:21.784Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:21.823Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:21.879Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:21.922Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:21.967Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.011Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.045Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.076Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.125Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.170Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.204Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.244Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.277Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.324Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.362Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.401Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.429Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.457Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.490Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.523Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.554Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.583Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.622Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.666Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.705Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.735Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.778Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:22.814Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:23.289Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:23.329Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:23.368Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:23.391Z: Starting 4 workers in us-central1-b...
Nov 17, 2018 1:10:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:23.725Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 17, 2018 1:10:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:10:39.265Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 17, 2018 1:11:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:11:02.873Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 17, 2018 1:11:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:11:02.911Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 17, 2018 1:11:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:11:05.364Z: Workers have started successfully.
Nov 17, 2018 1:11:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:11:05.400Z: Workers have started successfully.
Nov 17, 2018 1:11:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:11:09.013Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 17, 2018 1:11:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:11:09.050Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 17, 2018 1:11:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:11:20.668Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 17, 2018 1:12:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:12:24.135Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 17, 2018 1:12:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:12:24.308Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 17, 2018 1:12:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:12:26.290Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 17, 2018 1:12:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:12:26.386Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 17, 2018 1:12:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:12:27.822Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 17, 2018 1:12:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:12:27.903Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 17, 2018 1:12:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:12:31.166Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 17, 2018 1:12:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:12:31.397Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 17, 2018 1:12:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:12:56.882Z: Cleaning up.
Nov 17, 2018 1:12:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T01:12:57.080Z: Stopping worker pool...
Nov 17, 2018 1:14:14 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-11-17T01:14:14.382Z Waiting for main pipeline to 'finish'
2018-11-17T01:14:14.449Z RUNNING Query7
2018-11-17T01:14:16.384Z new perf {"snapshots":null,"runtimeSec":97.728,"numEvents":10000000,"eventsPerSec":102324.81990831696,"eventBytesPerSec":1.3134054692616241E7,"numResults":100,"resultsPerSec":1.0232481990831697,"resultBytesPerSec":103.37876555337263,"startupDelaySec":-175.836,"processingDelaySec":74.507,"shutdownDelaySec":78.108,"timeDilation":10.062622789783891,"errors":null,"jobId":null}
2018-11-17T01:14:31.447Z RUNNING Query7
2018-11-17T01:14:32.822Z no activity
2018-11-17T01:14:47.872Z RUNNING Query7
2018-11-17T01:14:49.296Z no activity
2018-11-17T01:15:04.395Z RUNNING Query7
2018-11-17T01:15:05.491Z no activity
2018-11-17T01:15:20.548Z RUNNING Query7
2018-11-17T01:15:22.112Z no activity
2018-11-17T01:15:37.179Z RUNNING Query7
2018-11-17T01:15:38.533Z no activity
2018-11-17T01:15:53.603Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 6,5,main]) completed. Took 6 mins 6.062 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 48s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/anid5bvtui5h4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1062
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1062/display/redirect?page=changes>
Changes:
[valentyn] Change Dataflow container prefix.
------------------------------------------
[...truncated 263.26 KB...]
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 17, 2018 12:42:21 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-JO3tp5zh08qhRSRIYeaasw.jar
Nov 17, 2018 12:42:23 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-JO3tp5zh08qhRSRIYeaasw.jar
Nov 17, 2018 12:42:26 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 17, 2018 12:42:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88173 bytes, hash 00eU-nqMeGyRKmEGydrCEA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-00eU-nqMeGyRKmEGydrCEA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 17, 2018 12:42:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-16_16_42_27-9790361880214550457
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_16_42_27-9790361880214550457?project=apache-beam-testing
Nov 17, 2018 12:42:28 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_16_42_27-9790361880214550457
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-17T00:42:27.817Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:30.972Z: Checking permissions granted to controller Service Account.
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:35.235Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:35.698Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:35.808Z: Expanding GroupByKey operations into optimizable parts.
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:35.837Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.036Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.088Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.124Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.190Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.248Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.342Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.377Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.425Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.488Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.532Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.585Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.627Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.672Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.724Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.771Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.819Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.888Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.938Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:36.994Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:37.048Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:37.097Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:37.147Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:37.203Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:37.250Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:37.301Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:37.354Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:37.400Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:37.436Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:37.484Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:37.539Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:37.587Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:37.635Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:37.678Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:37.730Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:38.261Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:38.315Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:38.363Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:38.363Z: Starting 4 workers in us-central1-b...
Nov 17, 2018 12:42:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:38.687Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 17, 2018 12:42:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:42:50.758Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 17, 2018 12:43:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:43:22.411Z: Workers have started successfully.
Nov 17, 2018 12:43:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:43:31.059Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 17, 2018 12:43:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:43:31.107Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 17, 2018 12:43:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:43:36.788Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 17, 2018 12:43:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:43:36.839Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 17, 2018 12:43:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:43:42.734Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 17, 2018 12:43:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:43:53.678Z: Workers have started successfully.
Nov 17, 2018 12:44:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:44:45.531Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 17, 2018 12:44:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:44:45.637Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 17, 2018 12:44:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:44:47.343Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 17, 2018 12:44:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:44:47.443Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 17, 2018 12:44:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:44:48.949Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 17, 2018 12:44:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:44:49.047Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 17, 2018 12:44:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:44:52.112Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 17, 2018 12:44:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:44:52.336Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 17, 2018 12:45:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:45:17.864Z: Cleaning up.
Nov 17, 2018 12:45:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:45:17.966Z: Stopping worker pool...
Nov 17, 2018 12:46:28 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-17T00:46:28.879Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-17T00:46:28.914Z RUNNING Query7
2018-11-17T00:46:30.276Z new perf {"snapshots":null,"runtimeSec":101.063,"numEvents":10000000,"eventsPerSec":98948.1808376953,"eventBytesPerSec":1.2700641154527374E7,"numResults":100,"resultsPerSec":0.989481808376953,"resultBytesPerSec":99.96734710032356,"startupDelaySec":-172.582,"processingDelaySec":77.831,"shutdownDelaySec":71.519,"timeDilation":9.730564103578955,"errors":null,"jobId":null}
2018-11-17T00:46:45.322Z RUNNING Query7
2018-11-17T00:46:46.400Z no activity
2018-11-17T00:47:01.442Z RUNNING Query7
2018-11-17T00:47:02.837Z no activity
2018-11-17T00:47:17.874Z RUNNING Query7
2018-11-17T00:47:18.984Z no activity
2018-11-17T00:47:34.023Z RUNNING Query7
2018-11-17T00:47:35.094Z no activity
2018-11-17T00:47:50.133Z RUNNING Query7
2018-11-17T00:47:51.197Z no activity
2018-11-17T00:48:06.259Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 9,5,main]) completed. Took 6 mins 7.476 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 55s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/73tsiydiydlcc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1061
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1061/display/redirect>
------------------------------------------
[...truncated 246.39 KB...]
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-tw3-RdAmWyXBDykBSqYpJg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-tw3-RdAmWyXBDykBSqYpJg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 17, 2018 12:04:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-tw3-RdAmWyXBDykBSqYpJg.jar
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 17, 2018 12:04:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 17, 2018 12:04:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 17, 2018 12:04:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 17, 2018 12:04:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 17, 2018 12:04:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 17, 2018 12:04:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 17, 2018 12:04:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 17, 2018 12:04:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 17, 2018 12:04:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 17, 2018 12:04:43 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 17, 2018 12:04:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88171 bytes, hash coLVp5_mA3dwPmGNNT8ubQ> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-coLVp5_mA3dwPmGNNT8ubQ.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-16_16_04_43-7085798344793310889
Nov 17, 2018 12:04:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_16_04_43-7085798344793310889?project=apache-beam-testing
Nov 17, 2018 12:04:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_16_04_43-7085798344793310889
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-17T00:04:43.547Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:46.447Z: Checking permissions granted to controller Service Account.
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:53.432Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:54.087Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:54.200Z: Expanding GroupByKey operations into optimizable parts.
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:54.250Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:54.467Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:54.501Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:54.539Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:54.576Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:54.621Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:54.680Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:54.728Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:54.780Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:54.834Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:54.883Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:54.923Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:54.958Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.006Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.048Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.083Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.131Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.176Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.226Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.266Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.314Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.357Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.403Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.452Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.503Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.538Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.588Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.650Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.700Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.750Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.803Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.853Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.904Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.957Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:55.986Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:56.502Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:56.543Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:56.584Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:56.606Z: Starting 4 workers in us-central1-b...
Nov 17, 2018 12:05:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:04:56.886Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 17, 2018 12:05:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:05:07.081Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 17, 2018 12:05:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:05:33.092Z: Workers have started successfully.
Nov 17, 2018 12:05:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:05:35.458Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 17, 2018 12:05:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:05:35.513Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 17, 2018 12:05:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:05:41.623Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 17, 2018 12:06:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:06:04.510Z: Workers have started successfully.
Nov 17, 2018 12:06:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:06:47.548Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 17, 2018 12:06:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:06:47.676Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 17, 2018 12:06:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:06:50.141Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 17, 2018 12:06:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:06:50.251Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 17, 2018 12:06:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:06:50.688Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 17, 2018 12:06:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:06:50.785Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 17, 2018 12:06:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:06:54.493Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 17, 2018 12:06:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:06:54.723Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 17, 2018 12:07:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:07:21.315Z: Cleaning up.
Nov 17, 2018 12:07:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-17T00:07:21.417Z: Stopping worker pool...
Nov 17, 2018 12:08:44 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-17T00:08:44.434Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-17T00:08:44.472Z RUNNING Query7
2018-11-17T00:08:45.767Z new perf {"snapshots":null,"runtimeSec":95.351,"numEvents":10000000,"eventsPerSec":104875.66989334144,"eventBytesPerSec":1.346147284244528E7,"numResults":100,"resultsPerSec":1.0487566989334145,"resultBytesPerSec":105.95588929324286,"startupDelaySec":-178.921,"processingDelaySec":70.814,"shutdownDelaySec":83.57,"timeDilation":10.313473377311198,"errors":null,"jobId":null}
2018-11-17T00:09:00.810Z RUNNING Query7
2018-11-17T00:09:01.997Z no activity
2018-11-17T00:09:17.034Z RUNNING Query7
2018-11-17T00:09:18.137Z no activity
2018-11-17T00:09:33.180Z RUNNING Query7
2018-11-17T00:09:34.276Z no activity
2018-11-17T00:09:49.335Z RUNNING Query7
2018-11-17T00:09:50.540Z no activity
2018-11-17T00:10:05.608Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 10,5,main]) completed. Took 5 mins 44.416 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 56s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/6dozdtqygusam
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1060
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1060/display/redirect?page=changes>
Changes:
[github] Add 2.7.0 LTS information to the docs. (#7069)
------------------------------------------
[...truncated 271.06 KB...]
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 16, 2018 9:22:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-x8ETaxBWaTY-4jW1I4soQg.jar
Nov 16, 2018 9:22:41 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-x8ETaxBWaTY-4jW1I4soQg.jar
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 16, 2018 9:22:45 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88191 bytes, hash mM9tTP1S6kkmxeyb2UdolA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-mM9tTP1S6kkmxeyb2UdolA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-16_13_22_46-13948597711480235458
Nov 16, 2018 9:22:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_13_22_46-13948597711480235458?project=apache-beam-testing
Nov 16, 2018 9:22:47 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_13_22_46-13948597711480235458
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-16T21:22:46.152Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:48.685Z: Checking permissions granted to controller Service Account.
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:53.865Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:54.380Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:54.454Z: Expanding GroupByKey operations into optimizable parts.
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:54.503Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:54.706Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:54.747Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#8bc528a076ecd5d7
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:54.795Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:54.828Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:54.865Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:54.915Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:54.964Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.011Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.038Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.089Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.139Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.181Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.229Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.275Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.312Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.353Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.391Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.439Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.486Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.528Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.567Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.617Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.666Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.704Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.754Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.791Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.832Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.862Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.896Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.932Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:55.971Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:56.018Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:56.080Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:56.119Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:56.509Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:56.535Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:56.570Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:56.597Z: Starting 4 workers in us-central1-b...
Nov 16, 2018 9:22:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:22:56.810Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 16, 2018 9:23:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:23:13.669Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 16, 2018 9:23:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:23:43.758Z: Workers have started successfully.
Nov 16, 2018 9:23:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:23:48.002Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 16, 2018 9:23:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:23:48.039Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 16, 2018 9:23:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:23:53.960Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 16, 2018 9:24:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:24:15.166Z: Workers have started successfully.
Nov 16, 2018 9:25:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:25:03.315Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 16, 2018 9:25:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:25:03.397Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 16, 2018 9:25:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:25:05.289Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 16, 2018 9:25:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:25:05.366Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 16, 2018 9:25:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:25:05.814Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 16, 2018 9:25:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:25:05.903Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 16, 2018 9:25:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:25:09.059Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 16, 2018 9:25:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:25:09.217Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 16, 2018 9:25:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:25:35.758Z: Cleaning up.
Nov 16, 2018 9:25:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:25:35.935Z: Stopping worker pool...
Nov 16, 2018 9:26:47 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-16T21:26:47.109Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-16T21:26:47.181Z RUNNING Query7
2018-11-16T21:26:48.596Z new perf {"snapshots":null,"runtimeSec":96.516,"numEvents":10000000,"eventsPerSec":103609.76418417672,"eventBytesPerSec":1.3298985629325707E7,"numResults":100,"resultsPerSec":1.0360976418417671,"resultBytesPerSec":104.67694475527374,"startupDelaySec":-168.946,"processingDelaySec":73.663,"shutdownDelaySec":72.43,"timeDilation":10.188984209871938,"errors":null,"jobId":null}
2018-11-16T21:27:03.671Z RUNNING Query7
2018-11-16T21:27:05.464Z no activity
2018-11-16T21:27:20.507Z RUNNING Query7
2018-11-16T21:27:21.475Z no activity
2018-11-16T21:27:36.521Z RUNNING Query7
2018-11-16T21:27:37.562Z no activity
2018-11-16T21:27:52.614Z RUNNING Query7
2018-11-16T21:27:53.729Z no activity
2018-11-16T21:28:08.761Z RUNNING Query7
2018-11-16T21:28:09.751Z no activity
2018-11-16T21:28:24.844Z RUNNING Query7
2018-11-16T21:28:25.069Z no activity
2018-11-16T21:28:40.070Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 7,5,main]) completed. Took 6 mins 26.804 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 16s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/b6hqepyhs6htq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1059
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1059/display/redirect?page=changes>
Changes:
[github] Fix possible internal dependency conflicts
------------------------------------------
[...truncated 236.20 KB...]
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5gNIY4GoYjxCFgfKFq71wA.jar
Nov 16, 2018 9:06:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5gNIY4GoYjxCFgfKFq71wA.jar
Nov 16, 2018 9:06:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5gNIY4GoYjxCFgfKFq71wA.jar
Nov 16, 2018 9:06:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5gNIY4GoYjxCFgfKFq71wA.jar
Nov 16, 2018 9:06:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5gNIY4GoYjxCFgfKFq71wA.jar
Nov 16, 2018 9:06:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5gNIY4GoYjxCFgfKFq71wA.jar
Nov 16, 2018 9:06:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5gNIY4GoYjxCFgfKFq71wA.jar
Nov 16, 2018 9:06:44 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5gNIY4GoYjxCFgfKFq71wA.jar
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 16, 2018 9:06:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88172 bytes, hash 6Y04WFXL_Jb1GKN0zGF7Zg> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-6Y04WFXL_Jb1GKN0zGF7Zg.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 16, 2018 9:06:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-16_13_06_49-14008648813800789237
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_13_06_49-14008648813800789237?project=apache-beam-testing
Nov 16, 2018 9:06:50 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_13_06_49-14008648813800789237
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-16T21:06:49.444Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:06:52.832Z: Checking permissions granted to controller Service Account.
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:06:59.150Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:06:59.647Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:06:59.739Z: Expanding GroupByKey operations into optimizable parts.
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:06:59.789Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.029Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.096Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#6044f02ac45a6f71
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.160Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.198Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.239Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.285Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.334Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.388Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.425Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.474Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.519Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.569Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.607Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.644Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.689Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.741Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.790Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.835Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.890Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.942Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:00.985Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:01.037Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:01.088Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:01.140Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:01.187Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:01.241Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:01.288Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:01.331Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:01.383Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:01.420Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:01.473Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:01.520Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:01.597Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:01.638Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:02.135Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:02.178Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:02.221Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:02.241Z: Starting 4 workers in us-central1-b...
Nov 16, 2018 9:07:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:02.549Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 16, 2018 9:07:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:14.321Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 16, 2018 9:07:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:42.110Z: Workers have started successfully.
Nov 16, 2018 9:07:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:48.461Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 16, 2018 9:07:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:48.493Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 16, 2018 9:07:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:54.755Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 16, 2018 9:07:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:07:54.805Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 16, 2018 9:08:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:08:00.512Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 16, 2018 9:08:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:08:12.281Z: Workers have started successfully.
Nov 16, 2018 9:09:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:09:02.256Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 16, 2018 9:09:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:09:02.374Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 16, 2018 9:09:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:09:03.745Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 16, 2018 9:09:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:09:03.890Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 16, 2018 9:09:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:09:04.486Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 16, 2018 9:09:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:09:04.635Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 16, 2018 9:09:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:09:08.553Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 16, 2018 9:09:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:09:08.999Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 16, 2018 9:09:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:09:32.570Z: Cleaning up.
Nov 16, 2018 9:09:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T21:09:32.703Z: Stopping worker pool...
Nov 16, 2018 9:10:50 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-16T21:10:50.561Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-16T21:10:50.615Z RUNNING Query7
2018-11-16T21:10:52.143Z new perf {"snapshots":null,"runtimeSec":93.963,"numEvents":10000000,"eventsPerSec":106424.86936347287,"eventBytesPerSec":1.366032264827645E7,"numResults":100,"resultsPerSec":1.0642486936347286,"resultBytesPerSec":107.52104551791663,"startupDelaySec":-172.283,"processingDelaySec":73.151,"shutdownDelaySec":78.321,"timeDilation":10.465821653203921,"errors":null,"jobId":null}
2018-11-16T21:11:07.188Z RUNNING Query7
2018-11-16T21:11:08.294Z no activity
2018-11-16T21:11:23.342Z RUNNING Query7
2018-11-16T21:11:24.961Z no activity
2018-11-16T21:11:40.082Z RUNNING Query7
2018-11-16T21:11:41.432Z no activity
2018-11-16T21:11:56.500Z RUNNING Query7
2018-11-16T21:11:58.143Z no activity
2018-11-16T21:12:13.247Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 10,5,main]) completed. Took 5 mins 32.91 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 21s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/td4w3mmdwz562
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1058
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1058/display/redirect?page=changes>
Changes:
[foegler] [BEAM-6049] Add option to load job to GCS in Dataflow Runner
[github] [BEAM-6073] Update README with lockfile & gradle instructions
------------------------------------------
[...truncated 254.56 KB...]
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 16, 2018 6:41:56 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-XTz_tSGTG8x1Hsdfy96YAA.jar
Nov 16, 2018 6:41:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-io-google-cloud-platform-2.9.0-SNAPSHOT-O1MLotkJ1jeG21lF7bNfDg.jar
Nov 16, 2018 6:41:59 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 115 files cached, 30 files newly uploaded
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 16, 2018 6:42:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88179 bytes, hash zFSDSGCAZ9Q5VotJEV6XCQ> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-zFSDSGCAZ9Q5VotJEV6XCQ.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-16_10_42_00-18420464209675948774
Nov 16, 2018 6:42:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_10_42_00-18420464209675948774?project=apache-beam-testing
Nov 16, 2018 6:42:01 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_10_42_00-18420464209675948774
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-16T18:42:00.822Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:03.980Z: Checking permissions granted to controller Service Account.
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:07.754Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:08.262Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:08.357Z: Expanding GroupByKey operations into optimizable parts.
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:08.410Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:08.579Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:08.623Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:08.662Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:08.707Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:08.757Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:08.799Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:08.851Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:08.892Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:08.937Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:08.985Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.030Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.085Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.134Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.183Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.227Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.282Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.330Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.378Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.418Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.467Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.506Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.551Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.602Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.654Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.704Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.756Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.797Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.833Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.871Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.917Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:09.963Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:10.011Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:10.054Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:10.106Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:10.637Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:10.687Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:10.729Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:10.751Z: Starting 4 workers in us-central1-b...
Nov 16, 2018 6:42:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:11.061Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 16, 2018 6:42:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:22.041Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 16, 2018 6:42:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:42:49.845Z: Workers have started successfully.
Nov 16, 2018 6:43:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:43:12.956Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 16, 2018 6:43:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:43:12.984Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 16, 2018 6:43:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:43:19.072Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 16, 2018 6:43:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:43:21.064Z: Workers have started successfully.
Nov 16, 2018 6:44:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:44:22.856Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 16, 2018 6:44:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:44:22.996Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 16, 2018 6:44:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:44:23.832Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 16, 2018 6:44:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:44:23.939Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 16, 2018 6:44:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:44:25.693Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 16, 2018 6:44:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:44:25.777Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 16, 2018 6:44:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:44:29.900Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 16, 2018 6:44:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:44:30.153Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 16, 2018 6:44:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:44:56.157Z: Cleaning up.
Nov 16, 2018 6:44:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:44:56.255Z: Stopping worker pool...
Nov 16, 2018 6:46:01 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-11-16T18:46:01.625Z Waiting for main pipeline to 'finish'
2018-11-16T18:46:01.703Z RUNNING Query7
2018-11-16T18:46:03.214Z new perf {"snapshots":null,"runtimeSec":95.485,"numEvents":10000000,"eventsPerSec":104728.49138608159,"eventBytesPerSec":1.344258152589412E7,"numResults":100,"resultsPerSec":1.0472849138608158,"resultBytesPerSec":105.80719484735822,"startupDelaySec":-162.136,"processingDelaySec":72.105,"shutdownDelaySec":66.651,"timeDilation":10.298999842907262,"errors":null,"jobId":null}
2018-11-16T18:46:18.308Z RUNNING Query7
2018-11-16T18:46:19.453Z no activity
2018-11-16T18:46:34.507Z RUNNING Query7
2018-11-16T18:46:35.623Z no activity
2018-11-16T18:46:50.674Z RUNNING Query7
2018-11-16T18:46:51.678Z no activity
2018-11-16T18:47:06.712Z RUNNING Query7
2018-11-16T18:47:08.252Z no activity
2018-11-16T18:47:23.305Z RUNNING Query7
2018-11-16T18:47:24.377Z no activity
2018-11-16T18:47:39.420Z RUNNING Query7
2018-11-16T18:47:40.355Z no activity
2018-11-16T18:47:55.411Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 5,5,main]) completed. Took 6 mins 13.762 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 32s
76 actionable tasks: 76 executed
Publishing build scan...
https://gradle.com/s/mfnopon4onnwi
Stopped 1 worker daemon(s).
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1057
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1057/display/redirect?page=changes>
Changes:
[robert] [BEAM-6073] Remove use of assert pacakge
------------------------------------------
[...truncated 356.24 KB...]
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/MapElements/Map as step s3
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Monitor as step s4
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Snoop as step s5
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) as step s6
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s7
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) as step s8
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize as step s9
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) as step s10
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys as step s11
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) as step s12
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections as step s13
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView as step s14
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) as step s15
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/AsBid as step s16
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles as step s17
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Debug as step s18
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Stamp as step s19
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.Format as step s20
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.DevNull as step s21
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 16, 2018 6:16:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <58912 bytes, hash OwBpf_-3R-ZfRIu9B_JU4g> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-OwBpf_-3R-ZfRIu9B_JU4g.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 16, 2018 6:16:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-16_10_16_35-9872712514019500287
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_10_16_35-9872712514019500287?project=apache-beam-testing
Nov 16, 2018 6:16:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_10_16_35-9872712514019500287
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 16, 2018 6:16:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88178 bytes, hash 4Ugur-7HmXXa633NZTfZdQ> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-4Ugur-7HmXXa633NZTfZdQ.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-16T18:16:35.812Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:39.245Z: Checking permissions granted to controller Service Account.
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:43.732Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:44.430Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:44.746Z: Expanding GroupByKey operations into optimizable parts.
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:44.821Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.005Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.055Z: Fusing consumer JoinToFiles/JoinToFiles.Monitor into JoinToFiles.ReadBounded
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.107Z: Fusing consumer JoinToFiles/JoinToFiles.Snoop into JoinToFiles/JoinToFiles.Monitor
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.169Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) into JoinToFiles/JoinToFiles.Snoop
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.217Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/AsBid into JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.267Z: Fusing consumer JoinToFiles.Format into JoinToFiles/JoinToFiles.Stamp
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.329Z: Fusing consumer JoinToFiles.DevNull into JoinToFiles.Format
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.402Z: Fusing consumer JoinToFiles/JoinToFiles.Stamp into JoinToFiles/JoinToFiles.Debug
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.469Z: Fusing consumer JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles into JoinToFiles/JoinToFiles/justBids/AsBid
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.532Z: Fusing consumer JoinToFiles/JoinToFiles.Debug into JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.598Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.655Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.699Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.751Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) into NexmarkUtils.GenerateSideInputData/MapElements/Map
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.815Z: Fusing consumer NexmarkUtils.GenerateSideInputData/MapElements/Map into NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.874Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.922Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:45.982Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:46.709Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Create
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:46.769Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:46.862Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Create
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:46.862Z: Starting 4 workers in us-central1-b...
Nov 16, 2018 6:16:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:47.306Z: Executing operation NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)+NexmarkUtils.GenerateSideInputData/MapElements/Map+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 16, 2018 6:16:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:16:58.970Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 16, 2018 6:17:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:17:35.355Z: Workers have started successfully.
Nov 16, 2018 6:17:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:17:39.024Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 16, 2018 6:17:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:17:39.073Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 16, 2018 6:17:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:17:39.968Z: Workers have started successfully.
Nov 16, 2018 6:17:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:17:44.649Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 16, 2018 6:17:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:17:44.703Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 16, 2018 6:17:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:17:56.069Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 16, 2018 6:17:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:17:56.713Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 16, 2018 6:17:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:17:56.910Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write
Nov 16, 2018 6:18:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:18:04.225Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Close
Nov 16, 2018 6:18:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:18:04.298Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Close
Nov 16, 2018 6:18:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:18:04.413Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey)
Nov 16, 2018 6:18:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:18:04.497Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize)
Nov 16, 2018 6:18:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:18:13.692Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections
Nov 16, 2018 6:18:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:18:14.013Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView
Nov 16, 2018 6:18:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:18:14.348Z: Executing operation JoinToFiles.ReadBounded+JoinToFiles/JoinToFiles.Monitor+JoinToFiles/JoinToFiles.Snoop+JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)+JoinToFiles/JoinToFiles/justBids/AsBid+JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles+JoinToFiles/JoinToFiles.Debug+JoinToFiles/JoinToFiles.Stamp+JoinToFiles.Format+JoinToFiles.DevNull
Nov 16, 2018 6:19:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:19:17.407Z: Cleaning up.
Nov 16, 2018 6:19:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:19:17.567Z: Stopping worker pool...
Nov 16, 2018 6:20:36 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-11-16T18:20:36.772Z Waiting for main pipeline to 'finish'
2018-11-16T18:20:36.811Z RUNNING JoinToFiles
2018-11-16T18:20:38.049Z new perf {"snapshots":null,"runtimeSec":59.518,"numEvents":10000000,"eventsPerSec":168016.3984004839,"eventBytesPerSec":2.1565995110722806E7,"numResults":9200000,"resultsPerSec":154575.0865284452,"resultBytesPerSec":5509791.911690581,"startupDelaySec":-140.197,"processingDelaySec":0.88,"shutdownDelaySec":80.679,"timeDilation":16.800799758056385,"errors":null,"jobId":null}
2018-11-16T18:20:53.136Z RUNNING JoinToFiles
2018-11-16T18:20:55.098Z no activity
2018-11-16T18:21:10.157Z RUNNING JoinToFiles
2018-11-16T18:21:11.843Z no activity
2018-11-16T18:21:26.886Z RUNNING JoinToFiles
2018-11-16T18:21:28.066Z no activity
2018-11-16T18:21:43.119Z RUNNING JoinToFiles
2018-11-16T18:21:44.534Z no activity
2018-11-16T18:21:59.575Z RUNNING JoinToFiles
2018-11-16T18:22:01.145Z no activity
2018-11-16T18:22:16.240Z DONE JoinToFiles
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 10,5,main]) completed. Took 6 mins 9.496 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 6s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/i2aq45i3deeoc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1056
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1056/display/redirect>
------------------------------------------
[...truncated 226.44 KB...]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Nov 16, 2018 6:04:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-io-google-cloud-platform-2.9.0-SNAPSHOT-R8cq4usa2MVNyhaZ4qJz3A.jar
Nov 16, 2018 6:04:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-2.9.0-SNAPSHOT-cJcLGXrx4EupIh1VdGa4lA.jar
Nov 16, 2018 6:04:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-extensions-protobuf-2.9.0-SNAPSHOT-9BRbhpUUPcV4od00EyfwrA.jar
Nov 16, 2018 6:04:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-extensions-join-library-2.9.0-SNAPSHOT-5rQ_bGVmq2nB9FW6grzkdA.jar
Nov 16, 2018 6:04:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-test-utils-2.9.0-SNAPSHOT-Bi-ZseuH1BOgDLAU7UYB9w.jar
Nov 16, 2018 6:04:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-core-construction-java-2.9.0-SNAPSHOT-AzGq4UehhtalMOwhQ8QJqA.jar
Nov 16, 2018 6:04:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-extensions-sql-2.9.0-SNAPSHOT-yoMwA0vGj6VuQHKQMdVvTw.jar
Nov 16, 2018 6:04:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.9.0-SNAPSHOT-Ng1AcFsraP_i9aWr-Gr6Aw.jar
Nov 16, 2018 6:04:00 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/io/kafka/build/libs/beam-sdks-java-io-kafka-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-io-kafka-2.9.0-SNAPSHOT-8MRnVTMtjnVRcxxN9Dp49A.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/testing/nexmark/build/libs/beam-sdks-java-nexmark-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-nexmark-2.9.0-SNAPSHOT-nyzp3xzvIrgTVh0iBXTE_w.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-core-2.9.0-SNAPSHOT-WRb2t2hFVKdkX8wa2nPGHQ.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-direct-java-2.9.0-SNAPSHOT-b3qM1iIjpxKqXhzd0H1BrA.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-model-pipeline-2.9.0-SNAPSHOT-6jOXXqHkgx1mK8DksU8LCw.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/model/job-management/build/libs/beam-model-job-management-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-model-job-management-2.9.0-SNAPSHOT-SmlUqxPsTpj5lRt29Ta2oQ.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Ahz9vJbB5fpcw5PiyZ3h9w.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Ahz9vJbB5fpcw5PiyZ3h9w.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Ahz9vJbB5fpcw5PiyZ3h9w.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/vendor/grpc-v1_13_1/build/libs/beam-vendor-grpc-v1_13_1-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-vendor-grpc-v1_13_1-2.9.0-SNAPSHOT-stdsRRuVpvf9bgf1T8nozQ.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Ahz9vJbB5fpcw5PiyZ3h9w.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Ahz9vJbB5fpcw5PiyZ3h9w.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Ahz9vJbB5fpcw5PiyZ3h9w.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Ahz9vJbB5fpcw5PiyZ3h9w.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Ahz9vJbB5fpcw5PiyZ3h9w.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Ahz9vJbB5fpcw5PiyZ3h9w.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Ahz9vJbB5fpcw5PiyZ3h9w.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Ahz9vJbB5fpcw5PiyZ3h9w.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Ahz9vJbB5fpcw5PiyZ3h9w.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Ahz9vJbB5fpcw5PiyZ3h9w.jar
Nov 16, 2018 6:04:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Ahz9vJbB5fpcw5PiyZ3h9w.jar
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.ReadBounded as step s1
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource) as step s2
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/MapElements/Map as step s3
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Monitor as step s4
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Snoop as step s5
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) as step s6
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s7
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) as step s8
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize as step s9
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) as step s10
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys as step s11
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) as step s12
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections as step s13
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView as step s14
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) as step s15
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/AsBid as step s16
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles as step s17
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Debug as step s18
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Stamp as step s19
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.Format as step s20
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.DevNull as step s21
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 16, 2018 6:04:05 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <58908 bytes, hash l0QZDN2ze-PMH24vcKcUVQ> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-l0QZDN2ze-PMH24vcKcUVQ.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 16, 2018 6:04:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-16_10_04_06-9364503814685562134
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_10_04_06-9364503814685562134?project=apache-beam-testing
Nov 16, 2018 6:04:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_10_04_06-9364503814685562134
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-16T18:04:06.593Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:09.347Z: Checking permissions granted to controller Service Account.
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:13.973Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:14.685Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:14.855Z: Expanding GroupByKey operations into optimizable parts.
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:14.890Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:14.973Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:15.017Z: Fusing consumer JoinToFiles/JoinToFiles.Monitor into JoinToFiles.ReadBounded
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:15.048Z: Fusing consumer JoinToFiles/JoinToFiles.Snoop into JoinToFiles/JoinToFiles.Monitor
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:15.094Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) into JoinToFiles/JoinToFiles.Snoop
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:15.137Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/AsBid into JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:15.175Z: Fusing consumer JoinToFiles.Format into JoinToFiles/JoinToFiles.Stamp
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:15.277Z: Fusing consumer JoinToFiles.DevNull into JoinToFiles.Format
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:15.318Z: Fusing consumer JoinToFiles/JoinToFiles.Stamp into JoinToFiles/JoinToFiles.Debug
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:15.353Z: Fusing consumer JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles into JoinToFiles/JoinToFiles/justBids/AsBid
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:15.386Z: Fusing consumer JoinToFiles/JoinToFiles.Debug into JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:15.433Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:15.485Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:15.524Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:15.566Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) into NexmarkUtils.GenerateSideInputData/MapElements/Map
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:15.604Z: Fusing consumer NexmarkUtils.GenerateSideInputData/MapElements/Map into NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:15.643Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:15.676Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:15.706Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:16.137Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Create
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:16.175Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:16.212Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Create
Nov 16, 2018 6:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:16.225Z: Starting 4 workers in us-central1-b...
Nov 16, 2018 6:04:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:16.456Z: Executing operation NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)+NexmarkUtils.GenerateSideInputData/MapElements/Map+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 16, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:04:37.295Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 16, 2018 6:05:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:05:06.216Z: Workers have started successfully.
Nov 16, 2018 6:05:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:05:12.377Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 16, 2018 6:05:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:05:12.428Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 16, 2018 6:05:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:05:18.120Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 16, 2018 6:05:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:05:28.473Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 16, 2018 6:05:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:05:28.589Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write
Nov 16, 2018 6:05:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:05:35.051Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Close
Nov 16, 2018 6:05:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:05:35.136Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Close
Nov 16, 2018 6:05:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:05:35.192Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey)
Nov 16, 2018 6:05:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:05:35.243Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize)
Nov 16, 2018 6:05:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:05:36.488Z: Workers have started successfully.
Nov 16, 2018 6:05:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:05:41.960Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections
Nov 16, 2018 6:05:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:05:42.182Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView
Nov 16, 2018 6:05:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:05:42.397Z: Executing operation JoinToFiles.ReadBounded+JoinToFiles/JoinToFiles.Monitor+JoinToFiles/JoinToFiles.Snoop+JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)+JoinToFiles/JoinToFiles/justBids/AsBid+JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles+JoinToFiles/JoinToFiles.Debug+JoinToFiles/JoinToFiles.Stamp+JoinToFiles.Format+JoinToFiles.DevNull
Nov 16, 2018 6:06:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:06:45.740Z: Cleaning up.
Nov 16, 2018 6:06:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T18:06:45.856Z: Stopping worker pool...
Nov 16, 2018 6:08:07 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-16T18:08:07.596Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-16T18:08:07.670Z RUNNING JoinToFiles
2018-11-16T18:08:09.236Z new perf {"snapshots":null,"runtimeSec":61.871,"numEvents":10000000,"eventsPerSec":161626.61020510417,"eventBytesPerSec":2.0745824328037366E7,"numResults":9200000,"resultsPerSec":148696.48138869583,"resultBytesPerSec":5300250.440432512,"startupDelaySec":-144.047,"processingDelaySec":0.778,"shutdownDelaySec":82.176,"timeDilation":16.161852887459393,"errors":null,"jobId":null}
2018-11-16T18:08:24.294Z RUNNING JoinToFiles
2018-11-16T18:08:25.610Z no activity
2018-11-16T18:08:40.706Z RUNNING JoinToFiles
2018-11-16T18:08:43.143Z no activity
2018-11-16T18:08:58.207Z RUNNING JoinToFiles
2018-11-16T18:08:59.902Z no activity
2018-11-16T18:09:14.963Z RUNNING JoinToFiles
2018-11-16T18:09:16.513Z no activity
2018-11-16T18:09:31.604Z DONE JoinToFiles
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':',5,main]) completed. Took 5 mins 34.687 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 19s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/rku2s7m223paw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1055
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1055/display/redirect?page=changes>
Changes:
[scott] [BEAM-6080] Rollback PR/7049 for failing java post-commits
------------------------------------------
[...truncated 264.33 KB...]
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Nov 16, 2018 5:44:01 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar>
java.io.IOException: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-HGp8N0LEIO3lzPHM-sy04Q.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-HGp8N0LEIO3lzPHM-sy04Q.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:432)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:287)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-HGp8N0LEIO3lzPHM-sy04Q.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-HGp8N0LEIO3lzPHM-sy04Q.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 16, 2018 5:44:04 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-HGp8N0LEIO3lzPHM-sy04Q.jar
Nov 16, 2018 5:44:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-HGp8N0LEIO3lzPHM-sy04Q.jar
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.ReadBounded as step s1
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource) as step s2
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/MapElements/Map as step s3
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Monitor as step s4
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Snoop as step s5
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) as step s6
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s7
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) as step s8
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize as step s9
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) as step s10
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys as step s11
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) as step s12
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections as step s13
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView as step s14
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) as step s15
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/AsBid as step s16
Nov 16, 2018 5:44:09 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles as step s17
Nov 16, 2018 5:44:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Debug as step s18
Nov 16, 2018 5:44:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Stamp as step s19
Nov 16, 2018 5:44:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.Format as step s20
Nov 16, 2018 5:44:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.DevNull as step s21
Nov 16, 2018 5:44:10 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 16, 2018 5:44:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <58912 bytes, hash 4qH-14r1nfOQKaFa-aVmxA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-4qH-14r1nfOQKaFa-aVmxA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 16, 2018 5:44:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-16_09_44_10-7377313897004085476
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_09_44_10-7377313897004085476?project=apache-beam-testing
Nov 16, 2018 5:44:11 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_09_44_10-7377313897004085476
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-16T17:44:10.657Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:13.461Z: Checking permissions granted to controller Service Account.
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:20.477Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:21.125Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:21.315Z: Expanding GroupByKey operations into optimizable parts.
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:21.368Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:21.461Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:21.509Z: Fusing consumer JoinToFiles/JoinToFiles.Monitor into JoinToFiles.ReadBounded
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:21.550Z: Fusing consumer JoinToFiles/JoinToFiles.Snoop into JoinToFiles/JoinToFiles.Monitor
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:21.602Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) into JoinToFiles/JoinToFiles.Snoop
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:21.632Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/AsBid into JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:21.842Z: Fusing consumer JoinToFiles.Format into JoinToFiles/JoinToFiles.Stamp
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:21.885Z: Fusing consumer JoinToFiles.DevNull into JoinToFiles.Format
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:21.920Z: Fusing consumer JoinToFiles/JoinToFiles.Stamp into JoinToFiles/JoinToFiles.Debug
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:21.957Z: Fusing consumer JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles into JoinToFiles/JoinToFiles/justBids/AsBid
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:22.016Z: Fusing consumer JoinToFiles/JoinToFiles.Debug into JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:22.053Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:22.095Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:22.139Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:22.197Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) into NexmarkUtils.GenerateSideInputData/MapElements/Map
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:22.240Z: Fusing consumer NexmarkUtils.GenerateSideInputData/MapElements/Map into NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:22.289Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:22.339Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:22.386Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:22.919Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Create
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:22.960Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:23.008Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Create
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:23.029Z: Starting 4 workers in us-central1-b...
Nov 16, 2018 5:44:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:23.294Z: Executing operation NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)+NexmarkUtils.GenerateSideInputData/MapElements/Map+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 16, 2018 5:44:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:44:36.507Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 16, 2018 5:45:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:45:11.351Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 16, 2018 5:45:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:45:11.395Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 16, 2018 5:45:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:45:13.316Z: Workers have started successfully.
Nov 16, 2018 5:45:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:45:13.361Z: Workers have started successfully.
Nov 16, 2018 5:45:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:45:17.512Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 16, 2018 5:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:45:30.327Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 16, 2018 5:45:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:45:30.415Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write
Nov 16, 2018 5:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:45:37.497Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Close
Nov 16, 2018 5:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:45:37.549Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Close
Nov 16, 2018 5:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:45:37.596Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey)
Nov 16, 2018 5:45:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:45:37.646Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize)
Nov 16, 2018 5:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:45:45.626Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections
Nov 16, 2018 5:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:45:45.872Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView
Nov 16, 2018 5:45:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:45:46.141Z: Executing operation JoinToFiles.ReadBounded+JoinToFiles/JoinToFiles.Monitor+JoinToFiles/JoinToFiles.Snoop+JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)+JoinToFiles/JoinToFiles/justBids/AsBid+JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles+JoinToFiles/JoinToFiles.Debug+JoinToFiles/JoinToFiles.Stamp+JoinToFiles.Format+JoinToFiles.DevNull
Nov 16, 2018 5:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:46:47.127Z: Cleaning up.
Nov 16, 2018 5:46:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T17:46:47.253Z: Stopping worker pool...
Nov 16, 2018 5:48:11 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-11-16T17:48:11.552Z Waiting for main pipeline to 'finish'
2018-11-16T17:48:11.603Z RUNNING JoinToFiles
2018-11-16T17:48:12.800Z new perf {"snapshots":null,"runtimeSec":59.517,"numEvents":10000000,"eventsPerSec":168019.22139892803,"eventBytesPerSec":2.1566357460893527E7,"numResults":9200000,"resultsPerSec":154577.68368701378,"resultBytesPerSec":5509884.486785288,"startupDelaySec":-144.203,"processingDelaySec":0.794,"shutdownDelaySec":84.686,"timeDilation":16.801082043785808,"errors":null,"jobId":null}
2018-11-16T17:48:27.844Z RUNNING JoinToFiles
2018-11-16T17:48:28.969Z no activity
2018-11-16T17:48:44.009Z RUNNING JoinToFiles
2018-11-16T17:48:45.137Z no activity
2018-11-16T17:49:00.179Z RUNNING JoinToFiles
2018-11-16T17:49:01.208Z no activity
2018-11-16T17:49:16.258Z RUNNING JoinToFiles
2018-11-16T17:49:17.290Z no activity
2018-11-16T17:49:32.395Z DONE JoinToFiles
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 4,5,main]) completed. Took 5 mins 48.433 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 22s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/znfnyu32uqrhe
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1054
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1054/display/redirect?page=changes>
Changes:
[lcwik] [BEAM-5674] Add withKeyType to DataflowRunner.Deduplicate internals to
------------------------------------------
[...truncated 233.99 KB...]
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-M1s0yoZ_cDbL25G0r5cvCQ.jar
Nov 16, 2018 4:18:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-M1s0yoZ_cDbL25G0r5cvCQ.jar
Nov 16, 2018 4:18:57 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-M1s0yoZ_cDbL25G0r5cvCQ.jar
Nov 16, 2018 4:18:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-M1s0yoZ_cDbL25G0r5cvCQ.jar
Nov 16, 2018 4:18:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-M1s0yoZ_cDbL25G0r5cvCQ.jar
Nov 16, 2018 4:18:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-M1s0yoZ_cDbL25G0r5cvCQ.jar
Nov 16, 2018 4:18:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-M1s0yoZ_cDbL25G0r5cvCQ.jar
Nov 16, 2018 4:18:58 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-M1s0yoZ_cDbL25G0r5cvCQ.jar
Nov 16, 2018 4:19:07 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 16, 2018 4:19:08 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88180 bytes, hash _MW2HRriMpex7n6Y7dSfKg> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-_MW2HRriMpex7n6Y7dSfKg.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 16, 2018 4:19:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-16_08_19_08-2339822814085440642
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_08_19_08-2339822814085440642?project=apache-beam-testing
Nov 16, 2018 4:19:09 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_08_19_08-2339822814085440642
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-16T16:19:09.022Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:11.588Z: Checking permissions granted to controller Service Account.
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:17.666Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:18.148Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:18.299Z: Expanding GroupByKey operations into optimizable parts.
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:18.349Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:18.535Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:18.579Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#a70ce18896339064
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:18.630Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:18.689Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:18.740Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:18.791Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:18.839Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:18.887Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:18.926Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:18.965Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.015Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.064Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.108Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.145Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.197Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.240Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.283Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.318Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.367Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.420Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.463Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.508Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.562Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.595Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.642Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.694Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.743Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.783Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.835Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.875Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.922Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:19.973Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:20.014Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:20.051Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:20.518Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:20.568Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:20.607Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:20.623Z: Starting 4 workers in us-central1-b...
Nov 16, 2018 4:19:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:20.944Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 16, 2018 4:19:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:30.689Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 16, 2018 4:19:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:53.917Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 16, 2018 4:19:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:53.952Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 16, 2018 4:19:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:57.364Z: Workers have started successfully.
Nov 16, 2018 4:19:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:57.393Z: Workers have started successfully.
Nov 16, 2018 4:20:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:59.865Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 16, 2018 4:20:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:19:59.905Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 16, 2018 4:20:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:20:16.974Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 16, 2018 4:21:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:21:14.924Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 16, 2018 4:21:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:21:15.030Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 16, 2018 4:21:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:21:16.779Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 16, 2018 4:21:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:21:16.877Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 16, 2018 4:21:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:21:19.793Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 16, 2018 4:21:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:21:19.915Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 16, 2018 4:21:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:21:23.151Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 16, 2018 4:21:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:21:23.386Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 16, 2018 4:21:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:21:49.644Z: Cleaning up.
Nov 16, 2018 4:21:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T16:21:49.761Z: Stopping worker pool...
Nov 16, 2018 4:23:09 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-16T16:23:09.946Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-16T16:23:09.990Z RUNNING Query7
2018-11-16T16:23:11.107Z new perf {"snapshots":null,"runtimeSec":98.543,"numEvents":10000000,"eventsPerSec":101478.5423622175,"eventBytesPerSec":1.3025429477486985E7,"numResults":100,"resultsPerSec":1.014785423622175,"resultBytesPerSec":102.52377134854834,"startupDelaySec":-179.829,"processingDelaySec":74.743,"shutdownDelaySec":81.286,"timeDilation":9.97939985590047,"errors":null,"jobId":null}
2018-11-16T16:23:26.155Z RUNNING Query7
2018-11-16T16:23:27.218Z no activity
2018-11-16T16:23:42.267Z RUNNING Query7
2018-11-16T16:23:45.290Z no activity
2018-11-16T16:24:00.337Z RUNNING Query7
2018-11-16T16:24:01.915Z no activity
2018-11-16T16:24:16.990Z RUNNING Query7
2018-11-16T16:24:18.053Z no activity
2018-11-16T16:24:33.116Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':',5,main]) completed. Took 5 mins 40.282 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 43s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/hq5fpqr3iblvw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1053
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1053/display/redirect?page=changes>
Changes:
[fabien.rousseau] [BEAM-6078] Use builder's keyspace by default when using
------------------------------------------
[...truncated 241.92 KB...]
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-rGfC9iAKkDk9GZKVWvB3nA.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 16, 2018 2:55:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-rGfC9iAKkDk9GZKVWvB3nA.jar
Nov 16, 2018 2:55:50 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 16, 2018 2:55:51 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88181 bytes, hash ED6UQeULRW7SEvz8ZojOBg> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-ED6UQeULRW7SEvz8ZojOBg.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 16, 2018 2:55:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-16_06_55_51-14803453177296435601
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_06_55_51-14803453177296435601?project=apache-beam-testing
Nov 16, 2018 2:55:52 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_06_55_51-14803453177296435601
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-16T14:55:51.812Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:55:54.707Z: Checking permissions granted to controller Service Account.
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:01.826Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:02.308Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:02.392Z: Expanding GroupByKey operations into optimizable parts.
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:02.434Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:02.605Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:02.648Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#a70ce18896339064
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:02.689Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:02.719Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:02.760Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:02.791Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:02.823Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:02.857Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:02.882Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:02.924Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:02.961Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:02.996Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.035Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.077Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.117Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.157Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.193Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.234Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.267Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.309Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.350Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.391Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.435Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.482Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.520Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.555Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.591Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.623Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.654Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.689Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.723Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.757Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.788Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:03.830Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:04.276Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:04.299Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:04.326Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:04.365Z: Starting 4 workers in us-central1-b...
Nov 16, 2018 2:56:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:04.625Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 16, 2018 2:56:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:21.924Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 16, 2018 2:56:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:48.356Z: Workers have started successfully.
Nov 16, 2018 2:56:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:51.036Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 16, 2018 2:56:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:51.075Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 16, 2018 2:56:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:57.223Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 16, 2018 2:56:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:56:57.261Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 16, 2018 2:57:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:57:08.548Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 16, 2018 2:57:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:57:19.754Z: Workers have started successfully.
Nov 16, 2018 2:58:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:58:06.799Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 16, 2018 2:58:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:58:06.897Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 16, 2018 2:58:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:58:07.737Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 16, 2018 2:58:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:58:07.804Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 16, 2018 2:58:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:58:10.104Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 16, 2018 2:58:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:58:10.213Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 16, 2018 2:58:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:58:13.958Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 16, 2018 2:58:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:58:14.118Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 16, 2018 2:58:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:58:44.198Z: Cleaning up.
Nov 16, 2018 2:58:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T14:58:44.319Z: Stopping worker pool...
Nov 16, 2018 2:59:52 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-11-16T14:59:52.607Z Waiting for main pipeline to 'finish'
2018-11-16T14:59:52.648Z RUNNING Query7
2018-11-16T14:59:53.674Z new perf {"snapshots":null,"runtimeSec":101.481,"numEvents":10000000,"eventsPerSec":98540.61351385975,"eventBytesPerSec":1.2648327243523419E7,"numResults":100,"resultsPerSec":0.9854061351385974,"resultBytesPerSec":99.5555818330525,"startupDelaySec":-170.534,"processingDelaySec":74.087,"shutdownDelaySec":69.053,"timeDilation":9.690483932952967,"errors":null,"jobId":null}
2018-11-16T15:00:08.721Z RUNNING Query7
2018-11-16T15:00:09.755Z no activity
2018-11-16T15:00:24.792Z RUNNING Query7
2018-11-16T15:00:25.854Z no activity
2018-11-16T15:00:40.895Z RUNNING Query7
2018-11-16T15:00:42.091Z no activity
2018-11-16T15:00:57.127Z RUNNING Query7
2018-11-16T15:00:58.088Z no activity
2018-11-16T15:01:13.137Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 7,5,main]) completed. Took 5 mins 39.994 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 20s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/rpdtuugi4ko34
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1052
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1052/display/redirect>
------------------------------------------
[...truncated 265.01 KB...]
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Nov 16, 2018 12:05:07 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackageWithRetry
WARNING: Upload attempt failed, sleeping before retrying staging of package: <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar>
java.io.IOException: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-GgEff3fmU8tqrnni-i6dmw.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-GgEff3fmU8tqrnni-i6dmw.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:432)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:287)
at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:104)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 429,
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-GgEff3fmU8tqrnni-i6dmw.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-GgEff3fmU8tqrnni-i6dmw.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 16, 2018 12:05:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-GgEff3fmU8tqrnni-i6dmw.jar
Nov 16, 2018 12:05:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-GgEff3fmU8tqrnni-i6dmw.jar
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.ReadBounded as step s1
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource) as step s2
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/MapElements/Map as step s3
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Monitor as step s4
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Snoop as step s5
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) as step s6
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s7
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) as step s8
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize as step s9
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) as step s10
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys as step s11
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) as step s12
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections as step s13
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView as step s14
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) as step s15
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/AsBid as step s16
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles as step s17
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Debug as step s18
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Stamp as step s19
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.Format as step s20
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.DevNull as step s21
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 16, 2018 12:05:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <58919 bytes, hash Wh0fIsb_81yvD_xKUpBfRw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-Wh0fIsb_81yvD_xKUpBfRw.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 16, 2018 12:05:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-16_04_05_22-7051357784637362811
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_04_05_22-7051357784637362811?project=apache-beam-testing
Nov 16, 2018 12:05:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_04_05_22-7051357784637362811
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-16T12:05:22.287Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:25.458Z: Checking permissions granted to controller Service Account.
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:32.023Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:32.479Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:32.624Z: Expanding GroupByKey operations into optimizable parts.
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:32.655Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:32.757Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:32.793Z: Fusing consumer JoinToFiles/JoinToFiles.Monitor into JoinToFiles.ReadBounded
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:32.833Z: Fusing consumer JoinToFiles/JoinToFiles.Snoop into JoinToFiles/JoinToFiles.Monitor
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:32.873Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) into JoinToFiles/JoinToFiles.Snoop
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:32.920Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/AsBid into JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:32.955Z: Fusing consumer JoinToFiles.Format into JoinToFiles/JoinToFiles.Stamp
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:32.992Z: Fusing consumer JoinToFiles.DevNull into JoinToFiles.Format
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:33.029Z: Fusing consumer JoinToFiles/JoinToFiles.Stamp into JoinToFiles/JoinToFiles.Debug
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:33.066Z: Fusing consumer JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles into JoinToFiles/JoinToFiles/justBids/AsBid
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:33.105Z: Fusing consumer JoinToFiles/JoinToFiles.Debug into JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:33.132Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:33.167Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:33.203Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:33.241Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) into NexmarkUtils.GenerateSideInputData/MapElements/Map
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:33.282Z: Fusing consumer NexmarkUtils.GenerateSideInputData/MapElements/Map into NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:33.318Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:33.356Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:33.394Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:33.880Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Create
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:33.911Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:33.942Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Create
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:33.980Z: Starting 4 workers in us-central1-b...
Nov 16, 2018 12:05:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:34.235Z: Executing operation NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)+NexmarkUtils.GenerateSideInputData/MapElements/Map+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 16, 2018 12:05:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:05:45.685Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 16, 2018 12:06:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:06:15.324Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 16, 2018 12:06:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:06:15.343Z: Workers have started successfully.
Nov 16, 2018 12:06:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:06:15.368Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 16, 2018 12:06:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:06:27.667Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 16, 2018 12:06:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:06:45.088Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 16, 2018 12:06:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:06:45.206Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write
Nov 16, 2018 12:06:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:06:46.739Z: Workers have started successfully.
Nov 16, 2018 12:06:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:06:53.248Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Close
Nov 16, 2018 12:06:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:06:53.286Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Close
Nov 16, 2018 12:06:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:06:53.380Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey)
Nov 16, 2018 12:06:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:06:53.459Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize)
Nov 16, 2018 12:07:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:07:02.985Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections
Nov 16, 2018 12:07:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:07:03.327Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView
Nov 16, 2018 12:07:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:07:03.604Z: Executing operation JoinToFiles.ReadBounded+JoinToFiles/JoinToFiles.Monitor+JoinToFiles/JoinToFiles.Snoop+JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)+JoinToFiles/JoinToFiles/justBids/AsBid+JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles+JoinToFiles/JoinToFiles.Debug+JoinToFiles/JoinToFiles.Stamp+JoinToFiles.Format+JoinToFiles.DevNull
Nov 16, 2018 12:08:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:08:08.962Z: Cleaning up.
Nov 16, 2018 12:08:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T12:08:09.186Z: Stopping worker pool...
Nov 16, 2018 12:09:23 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-16T12:09:23.200Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-16T12:09:23.236Z RUNNING JoinToFiles
2018-11-16T12:09:24.331Z new perf {"snapshots":null,"runtimeSec":62.583,"numEvents":10000000,"eventsPerSec":159787.80179921066,"eventBytesPerSec":2.0509801335826024E7,"numResults":9200000,"resultsPerSec":147004.7776552738,"resultBytesPerSec":5239950.066311938,"startupDelaySec":-138.274,"processingDelaySec":0.805,"shutdownDelaySec":75.691,"timeDilation":15.97798124091207,"errors":null,"jobId":null}
2018-11-16T12:09:39.384Z RUNNING JoinToFiles
2018-11-16T12:09:40.573Z no activity
2018-11-16T12:09:55.613Z RUNNING JoinToFiles
2018-11-16T12:09:56.607Z no activity
2018-11-16T12:10:11.660Z RUNNING JoinToFiles
2018-11-16T12:10:12.951Z no activity
2018-11-16T12:10:28.001Z RUNNING JoinToFiles
2018-11-16T12:10:29.450Z no activity
2018-11-16T12:10:44.497Z RUNNING JoinToFiles
2018-11-16T12:10:45.615Z no activity
2018-11-16T12:11:00.661Z DONE JoinToFiles
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':',5,main]) completed. Took 6 mins 11.727 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 45s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/uutwavt4tt4su
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1051
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1051/display/redirect?page=changes>
Changes:
[github] Merge pull request #6985: [BEAM-5983] Create Combine load test
------------------------------------------
[...truncated 269.70 KB...]
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 16, 2018 9:44:54 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-6Z3sImouX2wF6xc-g3sTdg.jar
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 16, 2018 9:44:58 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 16, 2018 9:44:59 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88179 bytes, hash 9BLAqh1Uef5HZTOddBH5pg> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-9BLAqh1Uef5HZTOddBH5pg.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-16_01_44_59-216582329923339986
Nov 16, 2018 9:45:00 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-16_01_44_59-216582329923339986?project=apache-beam-testing
Nov 16, 2018 9:45:00 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-16_01_44_59-216582329923339986
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-16T09:44:59.588Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:02.808Z: Checking permissions granted to controller Service Account.
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:09.508Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.109Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.188Z: Expanding GroupByKey operations into optimizable parts.
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.231Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.378Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.417Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.456Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.502Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.540Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.573Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.614Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.654Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.684Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.725Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.765Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.805Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.843Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.878Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.918Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.944Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:10.967Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.011Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.047Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.079Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.107Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.147Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.185Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.226Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.261Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.309Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.352Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.393Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.419Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.450Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.485Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.522Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.551Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.582Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:11.979Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:12.020Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:12.061Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:12.061Z: Starting 4 workers in us-central1-b...
Nov 16, 2018 9:45:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:12.291Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 16, 2018 9:45:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:25.670Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 16, 2018 9:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:54.647Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 16, 2018 9:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:54.680Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 16, 2018 9:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:54.899Z: Workers have started successfully.
Nov 16, 2018 9:45:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:45:54.932Z: Workers have started successfully.
Nov 16, 2018 9:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:46:11.488Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 16, 2018 9:46:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:46:11.531Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 16, 2018 9:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:46:17.658Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 16, 2018 9:46:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:46:17.694Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 16, 2018 9:46:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:46:28.730Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 16, 2018 9:47:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:47:24.094Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 16, 2018 9:47:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:47:24.168Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 16, 2018 9:47:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:47:25.868Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 16, 2018 9:47:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:47:25.948Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 16, 2018 9:47:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:47:27.531Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 16, 2018 9:47:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:47:27.620Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 16, 2018 9:47:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:47:31.194Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 16, 2018 9:47:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:47:31.376Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 16, 2018 9:47:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:47:58.284Z: Cleaning up.
Nov 16, 2018 9:47:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T09:47:58.426Z: Stopping worker pool...
Nov 16, 2018 9:49:00 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-16T09:49:00.554Z Waiting for main pipeline to 'finish'WARNING: No terminal state was returned. State value RUNNING
2018-11-16T09:49:00.596Z RUNNING Query7
2018-11-16T09:49:01.720Z new perf {"snapshots":null,"runtimeSec":108.232,"numEvents":9500000,"eventsPerSec":87774.41052553774,"eventBytesPerSec":1.1266324525094243E7,"numResults":100,"resultsPerSec":0.923941163426713,"resultBytesPerSec":93.3457757410008,"startupDelaySec":-171.379,"processingDelaySec":83.834,"shutdownDelaySec":63.147,"timeDilation":9.086037401138295,"errors":null,"jobId":null}
2018-11-16T09:49:16.771Z RUNNING Query7
2018-11-16T09:49:17.731Z no activity
2018-11-16T09:49:32.766Z RUNNING Query7
2018-11-16T09:49:33.760Z no activity
2018-11-16T09:49:48.806Z RUNNING Query7
2018-11-16T09:49:49.890Z no activity
2018-11-16T09:50:04.937Z RUNNING Query7
2018-11-16T09:50:05.953Z no activity
2018-11-16T09:50:21.004Z RUNNING Query7
2018-11-16T09:50:21.901Z no activity
2018-11-16T09:50:36.946Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':',5,main]) completed. Took 6 mins 6.936 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 28s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/zhgweqdh6whao
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1050
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1050/display/redirect>
------------------------------------------
[...truncated 233.38 KB...]
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-2x-0A1UlP192zXvU48k_tQ.jar
Nov 16, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-2x-0A1UlP192zXvU48k_tQ.jar
Nov 16, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-2x-0A1UlP192zXvU48k_tQ.jar
Nov 16, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-2x-0A1UlP192zXvU48k_tQ.jar
Nov 16, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-2x-0A1UlP192zXvU48k_tQ.jar
Nov 16, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-2x-0A1UlP192zXvU48k_tQ.jar
Nov 16, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-2x-0A1UlP192zXvU48k_tQ.jar
Nov 16, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-2x-0A1UlP192zXvU48k_tQ.jar
Nov 16, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-2x-0A1UlP192zXvU48k_tQ.jar
Nov 16, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/vendor/grpc-v1_13_1/build/libs/beam-vendor-grpc-v1_13_1-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-vendor-grpc-v1_13_1-2.9.0-SNAPSHOT-CWOoLj76cZRWg4GIu3Lemg.jar
Nov 16, 2018 6:04:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-2x-0A1UlP192zXvU48k_tQ.jar
Nov 16, 2018 6:04:22 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 16, 2018 6:04:23 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 16, 2018 6:04:24 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88180 bytes, hash qWRn0CU7C_1_0Mhh6Qhldw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-qWRn0CU7C_1_0Mhh6Qhldw.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 16, 2018 6:04:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-15_22_04_24-1636405551156154180
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-15_22_04_24-1636405551156154180?project=apache-beam-testing
Nov 16, 2018 6:04:25 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-15_22_04_24-1636405551156154180
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-16T06:04:24.582Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:27.044Z: Checking permissions granted to controller Service Account.
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:33.989Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:34.620Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:34.750Z: Expanding GroupByKey operations into optimizable parts.
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:34.795Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:35.224Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:35.284Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#91992c3387500b6e
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:35.336Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:35.385Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:35.454Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:35.503Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:35.553Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:35.603Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:35.653Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:35.697Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:35.739Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:35.786Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:35.836Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:35.884Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:35.937Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:35.985Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.031Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.081Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.124Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.184Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.229Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.278Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.328Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.385Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.438Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.483Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.533Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.586Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.639Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.673Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.715Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.766Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.810Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:36.859Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:37.393Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:37.446Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:37.502Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:37.515Z: Starting 4 workers in us-central1-b...
Nov 16, 2018 6:04:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:37.883Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 16, 2018 6:04:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:04:49.054Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 16, 2018 6:05:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:05:16.286Z: Workers have started successfully.
Nov 16, 2018 6:05:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:05:19.198Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 16, 2018 6:05:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:05:46.493Z: Workers have started successfully.
Nov 16, 2018 6:06:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:06:31.166Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 16, 2018 6:06:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:06:31.290Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 16, 2018 6:06:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:06:32.647Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 16, 2018 6:06:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:06:32.748Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 16, 2018 6:06:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:06:35.335Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 16, 2018 6:06:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:06:35.438Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 16, 2018 6:06:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:06:38.804Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 16, 2018 6:06:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:06:39.025Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 16, 2018 6:07:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:07:06.694Z: Cleaning up.
Nov 16, 2018 6:07:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T06:07:06.823Z: Stopping worker pool...
Nov 16, 2018 6:08:25 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-16T06:08:25.312Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-16T06:08:25.346Z RUNNING Query7
2018-11-16T06:08:26.508Z new perf {"snapshots":null,"runtimeSec":96.625,"numEvents":10000000,"eventsPerSec":103492.8848641656,"eventBytesPerSec":1.3283983410090556E7,"numResults":100,"resultsPerSec":1.034928848641656,"resultBytesPerSec":104.55886157826649,"startupDelaySec":-175.961,"processingDelaySec":71.843,"shutdownDelaySec":79.336,"timeDilation":10.177490297542043,"errors":null,"jobId":null}
2018-11-16T06:08:41.559Z RUNNING Query7
2018-11-16T06:08:42.778Z no activity
2018-11-16T06:08:57.823Z RUNNING Query7
2018-11-16T06:08:58.893Z no activity
2018-11-16T06:09:13.929Z RUNNING Query7
2018-11-16T06:09:14.982Z no activity
2018-11-16T06:09:30.022Z RUNNING Query7
2018-11-16T06:09:31.013Z no activity
2018-11-16T06:09:46.056Z RUNNING Query7
2018-11-16T06:09:47.228Z no activity
2018-11-16T06:10:02.288Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 8,5,main]) completed. Took 5 mins 48.193 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 55s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/vpshqxaszr2o2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1049
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1049/display/redirect?page=changes>
Changes:
[markliu] [BEAM-5788] Update Dataflow API protocol buffers
------------------------------------------
[...truncated 242.74 KB...]
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-azaO7hBpWeU2nY2GsuRBaA.jar
Nov 16, 2018 4:39:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-azaO7hBpWeU2nY2GsuRBaA.jar
Nov 16, 2018 4:39:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-azaO7hBpWeU2nY2GsuRBaA.jar
Nov 16, 2018 4:39:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-azaO7hBpWeU2nY2GsuRBaA.jar
Nov 16, 2018 4:39:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-azaO7hBpWeU2nY2GsuRBaA.jar
Nov 16, 2018 4:39:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-azaO7hBpWeU2nY2GsuRBaA.jar
Nov 16, 2018 4:39:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-azaO7hBpWeU2nY2GsuRBaA.jar
Nov 16, 2018 4:39:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-azaO7hBpWeU2nY2GsuRBaA.jar
Nov 16, 2018 4:39:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-azaO7hBpWeU2nY2GsuRBaA.jar
Nov 16, 2018 4:39:36 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 16, 2018 4:39:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88176 bytes, hash Ctocma3AiiHlaSX2StF2oA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-Ctocma3AiiHlaSX2StF2oA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-15_20_39_37-6213371293514252400
Nov 16, 2018 4:39:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-15_20_39_37-6213371293514252400?project=apache-beam-testing
Nov 16, 2018 4:39:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-15_20_39_37-6213371293514252400
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-16T04:39:37.828Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:40.416Z: Checking permissions granted to controller Service Account.
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:47.868Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:48.324Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:48.396Z: Expanding GroupByKey operations into optimizable parts.
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:48.432Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:48.590Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:48.627Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#6acf25f56c3241c8
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:48.667Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:48.703Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:48.747Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:48.788Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:48.826Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:48.867Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:48.897Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:48.932Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:48.971Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.006Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.032Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.057Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.094Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.134Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.176Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.217Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.246Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.280Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.325Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.359Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.395Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.434Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.472Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.502Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.530Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.562Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.601Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.644Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.666Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.705Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.748Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:49.786Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:50.210Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:50.241Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:50.284Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:50.297Z: Starting 4 workers in us-central1-b...
Nov 16, 2018 4:39:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:39:50.509Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 16, 2018 4:40:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:40:01.774Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 16, 2018 4:40:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:40:29.566Z: Workers have started successfully.
Nov 16, 2018 4:40:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:40:30.928Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 16, 2018 4:40:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:40:30.959Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 16, 2018 4:40:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:40:31.813Z: Workers have started successfully.
Nov 16, 2018 4:40:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:40:36.915Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 16, 2018 4:41:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:41:43.611Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 16, 2018 4:41:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:41:43.684Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 16, 2018 4:41:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:41:46.009Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 16, 2018 4:41:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:41:46.088Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 16, 2018 4:41:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:41:47.531Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 16, 2018 4:41:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:41:47.615Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 16, 2018 4:41:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:41:51.060Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 16, 2018 4:41:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:41:51.231Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 16, 2018 4:42:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:42:15.737Z: Cleaning up.
Nov 16, 2018 4:42:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T04:42:15.844Z: Stopping worker pool...
Nov 16, 2018 4:43:38 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-16T04:43:38.542Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-16T04:43:38.577Z RUNNING Query7
2018-11-16T04:43:39.617Z new perf {"snapshots":null,"runtimeSec":92.633,"numEvents":10000000,"eventsPerSec":107952.8893590837,"eventBytesPerSec":1.3856453931104466E7,"numResults":100,"resultsPerSec":1.079528893590837,"resultBytesPerSec":109.06480411948226,"startupDelaySec":-175.911,"processingDelaySec":70.875,"shutdownDelaySec":83.278,"timeDilation":10.616087139572292,"errors":null,"jobId":null}
2018-11-16T04:43:54.651Z RUNNING Query7
2018-11-16T04:43:55.687Z no activity
2018-11-16T04:44:10.733Z RUNNING Query7
2018-11-16T04:44:11.711Z no activity
2018-11-16T04:44:26.790Z RUNNING Query7
2018-11-16T04:44:27.882Z no activity
2018-11-16T04:44:42.918Z RUNNING Query7
2018-11-16T04:44:43.820Z no activity
2018-11-16T04:44:58.871Z RUNNING Query7
2018-11-16T04:44:58.993Z no activity
2018-11-16T04:45:13.993Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':',5,main]) completed. Took 5 mins 45.7 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 57s
76 actionable tasks: 76 executed
Publishing build scan...
https://gradle.com/s/nwt7hqdipfgn6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1048
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1048/display/redirect?page=changes>
Changes:
[github] [BEAM-6056] Publish gRPC 1.13.1 as a vendored dependency. (#7042)
------------------------------------------
[...truncated 245.77 KB...]
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-I-fRCMFQobxcqphwtj1yIQ.jar
Nov 16, 2018 1:34:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-I-fRCMFQobxcqphwtj1yIQ.jar
Nov 16, 2018 1:34:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-I-fRCMFQobxcqphwtj1yIQ.jar
Nov 16, 2018 1:34:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-I-fRCMFQobxcqphwtj1yIQ.jar
Nov 16, 2018 1:34:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-I-fRCMFQobxcqphwtj1yIQ.jar
Nov 16, 2018 1:34:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-core-2.9.0-SNAPSHOT-l907-NQLJ-fyxxWGY1NleA.jar
Nov 16, 2018 1:34:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-I-fRCMFQobxcqphwtj1yIQ.jar
Nov 16, 2018 1:34:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-I-fRCMFQobxcqphwtj1yIQ.jar
Nov 16, 2018 1:34:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-I-fRCMFQobxcqphwtj1yIQ.jar
Nov 16, 2018 1:34:27 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-I-fRCMFQobxcqphwtj1yIQ.jar
Nov 16, 2018 1:34:31 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 16, 2018 1:34:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88164 bytes, hash RRIbV_ZDOQspuzihoORhEg> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-RRIbV_ZDOQspuzihoORhEg.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 16, 2018 1:34:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-15_17_34_32-16506366108906897400
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-15_17_34_32-16506366108906897400?project=apache-beam-testing
Nov 16, 2018 1:34:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-15_17_34_32-16506366108906897400
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-16T01:34:32.873Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:35.378Z: Checking permissions granted to controller Service Account.
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:41.222Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:41.825Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:41.907Z: Expanding GroupByKey operations into optimizable parts.
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:41.952Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:42.099Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:42.138Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#a70ce18896339064
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:42.173Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:42.210Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:42.575Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:42.599Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:42.624Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:42.658Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:42.691Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:42.734Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:42.785Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:42.835Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:42.869Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:42.909Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:42.944Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:42.971Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.004Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.052Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.093Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.134Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.169Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.194Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.223Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.261Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.292Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.323Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.359Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.402Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.444Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.480Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.511Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.541Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.580Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:43.622Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:44.019Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:44.063Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:44.099Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:44.130Z: Starting 4 workers in us-central1-b...
Nov 16, 2018 1:34:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:44.441Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 16, 2018 1:35:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:34:57.602Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 16, 2018 1:35:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:35:24.975Z: Workers have started successfully.
Nov 16, 2018 1:35:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:35:26.741Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 16, 2018 1:35:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:35:26.777Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 16, 2018 1:35:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:35:38.572Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 16, 2018 1:35:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:35:56.351Z: Workers have started successfully.
Nov 16, 2018 1:36:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:36:45.303Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 16, 2018 1:36:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:36:45.823Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 16, 2018 1:36:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:36:48.417Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 16, 2018 1:36:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:36:48.500Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 16, 2018 1:36:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:36:49.763Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 16, 2018 1:36:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:36:49.842Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 16, 2018 1:36:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:36:53.359Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 16, 2018 1:36:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:36:53.539Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 16, 2018 1:37:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:37:20.470Z: Cleaning up.
Nov 16, 2018 1:37:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T01:37:20.565Z: Stopping worker pool...
Nov 16, 2018 1:38:33 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-16T01:38:33.691Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-16T01:38:33.735Z RUNNING Query7
2018-11-16T01:38:35.283Z new perf {"snapshots":null,"runtimeSec":100.886,"numEvents":10000000,"eventsPerSec":99121.78102016138,"eventBytesPerSec":1.2722923864559999E7,"numResults":100,"resultsPerSec":0.9912178102016137,"resultBytesPerSec":100.14273536466904,"startupDelaySec":-174.637,"processingDelaySec":76.418,"shutdownDelaySec":73.751,"timeDilation":9.747635945522669,"errors":null,"jobId":null}
2018-11-16T01:38:50.382Z RUNNING Query7
2018-11-16T01:38:51.930Z no activity
2018-11-16T01:39:06.990Z RUNNING Query7
2018-11-16T01:39:09.021Z no activity
2018-11-16T01:39:24.064Z RUNNING Query7
2018-11-16T01:39:25.281Z no activity
2018-11-16T01:39:40.316Z RUNNING Query7
2018-11-16T01:39:41.432Z no activity
2018-11-16T01:39:56.551Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 11,5,main]) completed. Took 5 mins 34.727 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 25s
76 actionable tasks: 74 executed, 2 from cache
Publishing build scan...
https://gradle.com/s/nvyo5aervtqf6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1047
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1047/display/redirect?page=changes>
Changes:
[robinyqiu] Fix for failure in beam-sdks-python:docs: Sphinx cannot find
------------------------------------------
[...truncated 246.99 KB...]
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-WkIWe3xLEBOxkGzrKNqGFQ.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 16, 2018 12:20:16 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-WkIWe3xLEBOxkGzrKNqGFQ.jar
Nov 16, 2018 12:20:19 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 16, 2018 12:20:20 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88175 bytes, hash OHhqLH9WvjR5MHNgKueodQ> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-OHhqLH9WvjR5MHNgKueodQ.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-15_16_20_20-126124477874043224
Nov 16, 2018 12:20:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-15_16_20_20-126124477874043224?project=apache-beam-testing
Nov 16, 2018 12:20:21 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-15_16_20_20-126124477874043224
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-16T00:20:20.746Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:23.759Z: Checking permissions granted to controller Service Account.
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:29.093Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:29.611Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:29.707Z: Expanding GroupByKey operations into optimizable parts.
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:29.760Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:29.988Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.033Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#252dc4dcb6895b21
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.071Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.123Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.183Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.232Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.283Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.331Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.381Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.427Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.464Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.523Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.563Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.626Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.677Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.727Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.769Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.812Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.861Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.903Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:30.961Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:31.014Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:31.064Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:31.111Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:31.174Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:31.243Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:31.299Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:31.358Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:31.402Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:31.459Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:31.507Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:31.551Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:31.613Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:31.666Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:32.206Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:32.242Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:32.281Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:32.313Z: Starting 4 workers in us-central1-b...
Nov 16, 2018 12:20:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:32.682Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 16, 2018 12:20:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:20:44.235Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 16, 2018 12:21:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:21:08.561Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 16, 2018 12:21:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:21:08.598Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 16, 2018 12:21:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:21:11.573Z: Workers have started successfully.
Nov 16, 2018 12:21:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:21:11.641Z: Workers have started successfully.
Nov 16, 2018 12:21:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:21:25.681Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 16, 2018 12:21:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:21:25.729Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 16, 2018 12:21:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:21:31.877Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 16, 2018 12:22:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:22:37.378Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 16, 2018 12:22:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:22:37.506Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 16, 2018 12:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:22:40.271Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 16, 2018 12:22:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:22:40.380Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 16, 2018 12:22:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:22:41.840Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 16, 2018 12:22:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:22:41.961Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 16, 2018 12:22:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:22:45.293Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 16, 2018 12:22:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:22:45.535Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 16, 2018 12:23:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:23:11.883Z: Cleaning up.
Nov 16, 2018 12:23:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:23:12.029Z: Stopping worker pool...
Nov 16, 2018 12:24:21 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-11-16T00:24:21.717Z Waiting for main pipeline to 'finish'
2018-11-16T00:24:21.773Z RUNNING Query7
2018-11-16T00:24:23.727Z new perf {"snapshots":null,"runtimeSec":105.944,"numEvents":10000000,"eventsPerSec":94389.48878652873,"eventBytesPerSec":1.2115503445216341E7,"numResults":100,"resultsPerSec":0.9438948878652873,"resultBytesPerSec":95.36170052102997,"startupDelaySec":-176.895,"processingDelaySec":82.398,"shutdownDelaySec":70.951,"timeDilation":9.282262327267235,"errors":null,"jobId":null}
2018-11-16T00:24:38.786Z RUNNING Query7
2018-11-16T00:24:40.226Z no activity
2018-11-16T00:24:55.269Z RUNNING Query7
2018-11-16T00:24:57.249Z no activity
2018-11-16T00:25:12.282Z RUNNING Query7
2018-11-16T00:25:13.564Z no activity
2018-11-16T00:25:28.658Z RUNNING Query7
2018-11-16T00:25:30.081Z no activity
2018-11-16T00:25:45.135Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 4,5,main]) completed. Took 5 mins 43.145 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 44s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/kxlhjd47nx646
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1046
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1046/display/redirect>
------------------------------------------
[...truncated 233.53 KB...]
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-hHfNSoFqtRXfzR9H9xD_6w.jar
Nov 16, 2018 12:03:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-hHfNSoFqtRXfzR9H9xD_6w.jar
Nov 16, 2018 12:03:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-hHfNSoFqtRXfzR9H9xD_6w.jar
Nov 16, 2018 12:03:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-hHfNSoFqtRXfzR9H9xD_6w.jar
Nov 16, 2018 12:03:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-hHfNSoFqtRXfzR9H9xD_6w.jar
Nov 16, 2018 12:03:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-hHfNSoFqtRXfzR9H9xD_6w.jar
Nov 16, 2018 12:03:58 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-hHfNSoFqtRXfzR9H9xD_6w.jar
Nov 16, 2018 12:04:01 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 16, 2018 12:04:02 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88172 bytes, hash NjRHEx0M88V0xPlQCSxoVQ> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-NjRHEx0M88V0xPlQCSxoVQ.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 16, 2018 12:04:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-15_16_04_02-17645301541495770261
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-15_16_04_02-17645301541495770261?project=apache-beam-testing
Nov 16, 2018 12:04:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-15_16_04_02-17645301541495770261
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-16T00:04:03.061Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:05.875Z: Checking permissions granted to controller Service Account.
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:13.718Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:14.227Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:14.326Z: Expanding GroupByKey operations into optimizable parts.
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:14.386Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:14.658Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:14.711Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#252dc4dcb6895b21
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:14.756Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:14.811Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:14.870Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:14.923Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:14.971Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.016Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.065Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.108Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.155Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.202Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.264Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.320Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.368Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.426Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.480Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.523Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.573Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.625Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.672Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.723Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.780Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.827Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.875Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.915Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:15.969Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:16.020Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:16.073Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:16.125Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:16.171Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:16.232Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:16.279Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:16.330Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:16.928Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:16.987Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:17.040Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:17.052Z: Starting 4 workers in us-central1-b...
Nov 16, 2018 12:04:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:17.497Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 16, 2018 12:04:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:04:38.344Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 16, 2018 12:05:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:05:05.431Z: Workers have started successfully.
Nov 16, 2018 12:05:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:05:18.799Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 16, 2018 12:05:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:05:18.856Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 16, 2018 12:05:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:05:24.382Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 16, 2018 12:05:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:05:37.625Z: Workers have started successfully.
Nov 16, 2018 12:06:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:06:33.587Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 16, 2018 12:06:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:06:33.715Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 16, 2018 12:06:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:06:35.669Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 16, 2018 12:06:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:06:35.813Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 16, 2018 12:06:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:06:38.174Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 16, 2018 12:06:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:06:38.269Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 16, 2018 12:06:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:06:42.503Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 16, 2018 12:06:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:06:42.782Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 16, 2018 12:07:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:07:09.351Z: Cleaning up.
Nov 16, 2018 12:07:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-16T00:07:09.475Z: Stopping worker pool...
Nov 16, 2018 12:08:03 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-11-16T00:08:03.909Z Waiting for main pipeline to 'finish'
2018-11-16T00:08:03.961Z RUNNING Query7
2018-11-16T00:08:05.222Z new perf {"snapshots":null,"runtimeSec":101.314,"numEvents":10000000,"eventsPerSec":98703.0420277553,"eventBytesPerSec":1.2669175997394241E7,"numResults":100,"resultsPerSec":0.9870304202775531,"resultBytesPerSec":99.71968336064118,"startupDelaySec":-156.496,"processingDelaySec":79.028,"shutdownDelaySec":55.182,"timeDilation":9.706457153009456,"errors":null,"jobId":null}
2018-11-16T00:08:20.268Z RUNNING Query7
2018-11-16T00:08:21.621Z no activity
2018-11-16T00:08:36.668Z RUNNING Query7
2018-11-16T00:08:37.831Z no activity
2018-11-16T00:08:52.885Z RUNNING Query7
2018-11-16T00:08:54.042Z no activity
2018-11-16T00:09:09.100Z RUNNING Query7
2018-11-16T00:09:10.475Z no activity
2018-11-16T00:09:25.540Z RUNNING Query7
2018-11-16T00:09:26.940Z no activity
2018-11-16T00:09:41.988Z RUNNING Query7
2018-11-16T00:09:43.160Z no activity
2018-11-16T00:09:58.197Z RUNNING Query7
2018-11-16T00:09:59.482Z no activity
2018-11-16T00:10:14.482Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Daemon worker,5,main]) completed. Took 6 mins 21.089 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 4s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/7ce3acxhzqgti
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1045
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1045/display/redirect?page=changes>
Changes:
[aaltay] [BEAM-5939] Dedupe runner constants (#6976)
------------------------------------------
[...truncated 264.81 KB...]
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 15, 2018 9:50:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-J_KeF75Lh8Fulpi42n9MMQ.jar
Nov 15, 2018 9:51:06 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-J_KeF75Lh8Fulpi42n9MMQ.jar
Nov 15, 2018 9:51:09 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 15, 2018 9:51:10 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88171 bytes, hash 2NJG3voz65aCJSyFgDKi6A> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-2NJG3voz65aCJSyFgDKi6A.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-15_13_51_11-1134052399322767177
Nov 15, 2018 9:51:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-15_13_51_11-1134052399322767177?project=apache-beam-testing
Nov 15, 2018 9:51:12 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-15_13_51_11-1134052399322767177
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-15T21:51:11.328Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:14.183Z: Checking permissions granted to controller Service Account.
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:21.518Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.032Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.098Z: Expanding GroupByKey operations into optimizable parts.
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.136Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.287Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.306Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.345Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.369Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.400Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.438Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.464Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.502Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.540Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.572Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.602Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.638Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.667Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.704Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.743Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.782Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.819Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.857Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.894Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.932Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:22.971Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:23.011Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:23.041Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:23.080Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:23.118Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:23.157Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:23.194Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:23.234Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:23.265Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:23.304Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:23.343Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:23.380Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:23.417Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:23.456Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:23.819Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:23.841Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:23.866Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:23.896Z: Starting 4 workers in us-central1-b...
Nov 15, 2018 9:51:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:24.048Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 15, 2018 9:51:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:51:41.144Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 15, 2018 9:52:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:52:16.479Z: Workers have started successfully.
Nov 15, 2018 9:52:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:52:28.767Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 15, 2018 9:52:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:52:28.794Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 15, 2018 9:52:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:52:34.277Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 15, 2018 9:52:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:52:47.884Z: Workers have started successfully.
Nov 15, 2018 9:53:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:53:45.921Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 15, 2018 9:53:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:53:46.005Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 15, 2018 9:53:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:53:48.047Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 15, 2018 9:53:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:53:48.116Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 15, 2018 9:53:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:53:49.508Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 15, 2018 9:53:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:53:49.573Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 15, 2018 9:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:53:53.721Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 15, 2018 9:53:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:53:53.900Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 15, 2018 9:54:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:54:19.648Z: Cleaning up.
Nov 15, 2018 9:54:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T21:54:19.731Z: Stopping worker pool...
Nov 15, 2018 9:55:12 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-15T21:55:12.270Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-15T21:55:12.313Z RUNNING Query7
2018-11-15T21:55:13.501Z new perf {"snapshots":null,"runtimeSec":101.051,"numEvents":10000000,"eventsPerSec":98959.93112388793,"eventBytesPerSec":1.270214938001603E7,"numResults":100,"resultsPerSec":0.9895993112388793,"resultBytesPerSec":99.97921841446399,"startupDelaySec":-154.076,"processingDelaySec":78.317,"shutdownDelaySec":53.025,"timeDilation":9.73171962672314,"errors":null,"jobId":null}
2018-11-15T21:55:28.537Z RUNNING Query7
2018-11-15T21:55:29.699Z no activity
2018-11-15T21:55:44.731Z RUNNING Query7
2018-11-15T21:55:45.932Z no activity
2018-11-15T21:56:00.967Z RUNNING Query7
2018-11-15T21:56:02.088Z no activity
2018-11-15T21:56:17.132Z RUNNING Query7
2018-11-15T21:56:18.226Z no activity
2018-11-15T21:56:33.266Z RUNNING Query7
2018-11-15T21:56:34.475Z no activity
2018-11-15T21:56:49.520Z RUNNING Query7
2018-11-15T21:56:50.557Z no activity
2018-11-15T21:57:05.637Z RUNNING Query7
2018-11-15T21:57:06.736Z no activity
2018-11-15T21:57:21.778Z RUNNING Query7
2018-11-15T21:57:22.780Z no activity
2018-11-15T21:57:37.851Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 11,5,main]) completed. Took 6 mins 56.124 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 11m 32s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/xx7wudugncjig
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1044
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1044/display/redirect?page=changes>
Changes:
[ankurgoenka] [BEAM-4524] Use SHA256 for artifact validation
[ankurgoenka] [BEAM-4524] use hex encoding for artifact hash
------------------------------------------
[...truncated 248.67 KB...]
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 15, 2018 8:36:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-KF1FnMlg-mD9KL9yR-j9uA.jar
Nov 15, 2018 8:36:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-KF1FnMlg-mD9KL9yR-j9uA.jar
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 15, 2018 8:36:46 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 15, 2018 8:36:47 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88184 bytes, hash YdtHZuIBidnkxzVicjWV8g> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-YdtHZuIBidnkxzVicjWV8g.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 15, 2018 8:36:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-15_12_36_47-14235448690462250713
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-15_12_36_47-14235448690462250713?project=apache-beam-testing
Nov 15, 2018 8:36:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-15_12_36_47-14235448690462250713
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-15T20:36:47.680Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:50.782Z: Checking permissions granted to controller Service Account.
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:58.276Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:58.807Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:58.886Z: Expanding GroupByKey operations into optimizable parts.
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:58.912Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.065Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.105Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#5e4921c369c80b03
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.146Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.175Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.216Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.252Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.281Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.312Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.342Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.378Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.416Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.460Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.484Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.521Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.558Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.596Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.625Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.664Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.706Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.748Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.782Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.818Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.857Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.897Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.939Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:36:59.982Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:37:00.016Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:37:00.056Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:37:00.090Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:37:00.123Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:37:00.162Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:37:00.201Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:37:00.237Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:37:00.278Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:37:00.722Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:37:00.752Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:37:00.778Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:37:00.808Z: Starting 4 workers in us-central1-b...
Nov 15, 2018 8:37:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:37:01.031Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 15, 2018 8:37:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:37:13.497Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 15, 2018 8:37:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:37:43.783Z: Workers have started successfully.
Nov 15, 2018 8:37:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:37:54.573Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 15, 2018 8:37:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:37:54.618Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 15, 2018 8:38:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:38:00.472Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 15, 2018 8:38:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:38:13.979Z: Workers have started successfully.
Nov 15, 2018 8:39:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:39:08.225Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 15, 2018 8:39:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:39:08.305Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 15, 2018 8:39:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:39:09.806Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 15, 2018 8:39:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:39:09.877Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 15, 2018 8:39:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:39:12.242Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 15, 2018 8:39:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:39:12.335Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 15, 2018 8:39:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:39:16.581Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 15, 2018 8:39:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:39:16.791Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 15, 2018 8:39:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:39:44.669Z: Cleaning up.
Nov 15, 2018 8:39:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T20:39:44.813Z: Stopping worker pool...
Nov 15, 2018 8:40:49 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-15T20:40:49.121Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-15T20:40:49.165Z RUNNING Query7
2018-11-15T20:40:50.474Z new perf {"snapshots":null,"runtimeSec":102.576,"numEvents":10000000,"eventsPerSec":97488.69131180784,"eventBytesPerSec":1.2513306202230543E7,"numResults":100,"resultsPerSec":0.9748869131180784,"resultBytesPerSec":98.49282483231946,"startupDelaySec":-167.372,"processingDelaySec":77.279,"shutdownDelaySec":64.796,"timeDilation":9.587037903603182,"errors":null,"jobId":null}
2018-11-15T20:41:05.531Z RUNNING Query7
2018-11-15T20:41:06.654Z no activity
2018-11-15T20:41:21.687Z RUNNING Query7
2018-11-15T20:41:22.963Z no activity
2018-11-15T20:41:38.029Z RUNNING Query7
2018-11-15T20:41:39.222Z no activity
2018-11-15T20:41:54.263Z RUNNING Query7
2018-11-15T20:41:55.451Z no activity
2018-11-15T20:42:10.499Z RUNNING Query7
2018-11-15T20:42:11.728Z no activity
2018-11-15T20:42:26.775Z RUNNING Query7
2018-11-15T20:42:27.835Z no activity
2018-11-15T20:42:42.878Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 3,5,main]) completed. Took 6 mins 13.943 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 57s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/gpxwf7ihlywpk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1043
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1043/display/redirect>
------------------------------------------
[...truncated 244.10 KB...]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 15, 2018 6:04:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-r4vgaN1yo05pzDGuv6omOQ.jar
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 15, 2018 6:04:25 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88173 bytes, hash z4NFq-nIjzV6roROMFTJNA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-z4NFq-nIjzV6roROMFTJNA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 15, 2018 6:04:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-15_10_04_26-7699945112798254425
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-15_10_04_26-7699945112798254425?project=apache-beam-testing
Nov 15, 2018 6:04:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-15_10_04_26-7699945112798254425
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-15T18:04:26.402Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:29.213Z: Checking permissions granted to controller Service Account.
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:35.815Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:36.336Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:36.473Z: Expanding GroupByKey operations into optimizable parts.
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:36.522Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:36.744Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:36.791Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#84744397f8dc9244
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:36.840Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:36.892Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:36.926Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:36.978Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.025Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.080Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.129Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.179Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.226Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.270Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.315Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.374Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.423Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.473Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.522Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.569Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.619Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.664Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.714Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.766Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.810Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.859Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.915Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:37.958Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:38.000Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:38.050Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:38.103Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:38.154Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:38.197Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:38.250Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:38.285Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:38.325Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 15, 2018 6:04:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:38.876Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 15, 2018 6:04:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:38.918Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 15, 2018 6:04:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:38.972Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 15, 2018 6:04:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:38.982Z: Starting 4 workers in us-central1-b...
Nov 15, 2018 6:04:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:39.248Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 15, 2018 6:04:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:04:50.786Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 15, 2018 6:05:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:05:47.969Z: Workers have started successfully.
Nov 15, 2018 6:06:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:06:00.111Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 15, 2018 6:06:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:06:00.155Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 15, 2018 6:06:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:06:18.144Z: Workers have started successfully.
Nov 15, 2018 6:06:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:06:29.511Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 15, 2018 6:06:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:06:29.552Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 15, 2018 6:06:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:06:35.026Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 15, 2018 6:07:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:07:27.919Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 15, 2018 6:07:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:07:28.017Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 15, 2018 6:07:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:07:31.005Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 15, 2018 6:07:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:07:31.084Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 15, 2018 6:07:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:07:32.387Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 15, 2018 6:07:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:07:32.469Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 15, 2018 6:07:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:07:37.652Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 15, 2018 6:07:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:07:37.859Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 15, 2018 6:08:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:08:05.060Z: Cleaning up.
Nov 15, 2018 6:08:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T18:08:05.150Z: Stopping worker pool...
Nov 15, 2018 6:08:27 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-15T18:08:27.382Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-15T18:08:27.410Z RUNNING Query7
2018-11-15T18:08:28.663Z new perf {"snapshots":null,"runtimeSec":115.545,"numEvents":10000000,"eventsPerSec":86546.3672162361,"eventBytesPerSec":1.1108787892163226E7,"numResults":100,"resultsPerSec":0.865463672162361,"resultBytesPerSec":87.43779479856333,"startupDelaySec":-138.64,"processingDelaySec":90.526,"shutdownDelaySec":23.095,"timeDilation":8.510969752044657,"errors":null,"jobId":null}
2018-11-15T18:08:43.716Z RUNNING Query7
2018-11-15T18:08:44.895Z no activity
2018-11-15T18:08:59.925Z RUNNING Query7
2018-11-15T18:09:01.466Z no activity
2018-11-15T18:09:16.502Z RUNNING Query7
2018-11-15T18:09:17.718Z no activity
2018-11-15T18:09:32.745Z RUNNING Query7
2018-11-15T18:09:33.795Z no activity
2018-11-15T18:09:48.837Z RUNNING Query7
2018-11-15T18:09:49.907Z no activity
2018-11-15T18:10:04.944Z RUNNING Query7
2018-11-15T18:10:06.148Z no activity
2018-11-15T18:10:21.181Z RUNNING Query7
2018-11-15T18:10:22.638Z no activity
2018-11-15T18:10:37.681Z RUNNING Query7
2018-11-15T18:10:38.729Z no activity
2018-11-15T18:10:53.774Z RUNNING Query7
2018-11-15T18:10:54.865Z no activity
2018-11-15T18:11:09.921Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':',5,main]) completed. Took 7 mins 3.409 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 11m 3s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/tpsbf5fattmfc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1042
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1042/display/redirect?page=changes>
Changes:
[lukasz.gajowy] [BEAM-5906] Split tableSpec method
[lukasz.gajowy] [BEAM-5906] Use dedicated API in Nexmark to publish results to BigQuery
[lukasz.gajowy] [BEAM-5906] Change google-cloud-bigquery dependency version
------------------------------------------
[...truncated 234.17 KB...]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Nov 15, 2018 4:21:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/io/kafka/build/libs/beam-sdks-java-io-kafka-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-io-kafka-2.9.0-SNAPSHOT-5CVziM2ovYBgqNjLtCXCdA.jar
Nov 15, 2018 4:21:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-core-construction-java-2.9.0-SNAPSHOT-Au7r7qPKNLdfmNyjz8MgeQ.jar
Nov 15, 2018 4:21:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-io-google-cloud-platform-2.9.0-SNAPSHOT-L5015WtlRlmteLrZBT7Idg.jar
Nov 15, 2018 4:21:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-direct-java-2.9.0-SNAPSHOT-ms_A3K9RrxBBbTFkqdgXoQ.jar
Nov 15, 2018 4:21:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-extensions-sql-2.9.0-SNAPSHOT-WMVZDHuD0n3vzDuKAgFp_A.jar
Nov 15, 2018 4:21:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-extensions-protobuf-2.9.0-SNAPSHOT-DIVjpDelCOlDBi-KHflf8g.jar
Nov 15, 2018 4:21:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-2.9.0-SNAPSHOT-gkFJSCQfzbhkJCTQibsqEg.jar
Nov 15, 2018 4:21:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-extensions-join-library-2.9.0-SNAPSHOT-F0SFEf5VSgaDq7jNRE7ptg.jar
Nov 15, 2018 4:21:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/testing/nexmark/build/libs/beam-sdks-java-nexmark-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-nexmark-2.9.0-SNAPSHOT-5YrGYLoODqaaEKs7AIaxKg.jar
Nov 15, 2018 4:21:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-test-utils-2.9.0-SNAPSHOT-jW3IvMcm4WDJcK6xMBRGVw.jar
Nov 15, 2018 4:21:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.9.0-SNAPSHOT-UxwVhwH2iVmobkFkX2P6DA.jar
Nov 15, 2018 4:21:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-core-2.9.0-SNAPSHOT-Po5PaPNaPKSi8fUt78ah_g.jar
Nov 15, 2018 4:21:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-model-pipeline-2.9.0-SNAPSHOT-RsO4TY2ynfHpKpbrlUhelQ.jar
Nov 15, 2018 4:21:18 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/model/job-management/build/libs/beam-model-job-management-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-model-job-management-2.9.0-SNAPSHOT-kgLdh0hyhUG9-HGATDCcpQ.jar
Nov 15, 2018 4:21:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/vendor/grpc-v1_13_1/build/libs/beam-vendor-grpc-v1_13_1-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-vendor-grpc-v1_13_1-2.9.0-SNAPSHOT-gcAYR6x8X_kjydgt_tsNkA.jar
Nov 15, 2018 4:21:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-S3FRdgRT1vnIiENOYd4hBA.jar
Nov 15, 2018 4:21:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-S3FRdgRT1vnIiENOYd4hBA.jar
Nov 15, 2018 4:21:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-S3FRdgRT1vnIiENOYd4hBA.jar
Nov 15, 2018 4:21:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-S3FRdgRT1vnIiENOYd4hBA.jar
Nov 15, 2018 4:21:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-S3FRdgRT1vnIiENOYd4hBA.jar
Nov 15, 2018 4:21:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-S3FRdgRT1vnIiENOYd4hBA.jar
Nov 15, 2018 4:21:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-S3FRdgRT1vnIiENOYd4hBA.jar
Nov 15, 2018 4:21:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-S3FRdgRT1vnIiENOYd4hBA.jar
Nov 15, 2018 4:21:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-S3FRdgRT1vnIiENOYd4hBA.jar
Nov 15, 2018 4:21:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-S3FRdgRT1vnIiENOYd4hBA.jar
Nov 15, 2018 4:21:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-S3FRdgRT1vnIiENOYd4hBA.jar
Nov 15, 2018 4:21:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-S3FRdgRT1vnIiENOYd4hBA.jar
Nov 15, 2018 4:21:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-S3FRdgRT1vnIiENOYd4hBA.jar
Nov 15, 2018 4:21:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-S3FRdgRT1vnIiENOYd4hBA.jar
Nov 15, 2018 4:21:22 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 116 files cached, 29 files newly uploaded
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.ReadBounded as step s1
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource) as step s2
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/MapElements/Map as step s3
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Monitor as step s4
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Snoop as step s5
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) as step s6
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s7
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) as step s8
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize as step s9
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) as step s10
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys as step s11
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) as step s12
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections as step s13
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView as step s14
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) as step s15
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/AsBid as step s16
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles as step s17
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Debug as step s18
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Stamp as step s19
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.Format as step s20
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.DevNull as step s21
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 15, 2018 4:21:23 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <58919 bytes, hash oBTkxs3DdISxZq3m3VkuQw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-oBTkxs3DdISxZq3m3VkuQw.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-15_08_21_23-6693628808027684169
Nov 15, 2018 4:21:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-15_08_21_23-6693628808027684169?project=apache-beam-testing
Nov 15, 2018 4:21:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-15_08_21_23-6693628808027684169
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-15T16:21:24.058Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:26.899Z: Checking permissions granted to controller Service Account.
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:32.828Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:33.319Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:33.495Z: Expanding GroupByKey operations into optimizable parts.
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:33.536Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:33.634Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:33.680Z: Fusing consumer JoinToFiles/JoinToFiles.Monitor into JoinToFiles.ReadBounded
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:33.723Z: Fusing consumer JoinToFiles/JoinToFiles.Snoop into JoinToFiles/JoinToFiles.Monitor
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:33.759Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) into JoinToFiles/JoinToFiles.Snoop
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:33.805Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/AsBid into JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:33.848Z: Fusing consumer JoinToFiles.Format into JoinToFiles/JoinToFiles.Stamp
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:33.884Z: Fusing consumer JoinToFiles.DevNull into JoinToFiles.Format
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:33.910Z: Fusing consumer JoinToFiles/JoinToFiles.Stamp into JoinToFiles/JoinToFiles.Debug
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:33.945Z: Fusing consumer JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles into JoinToFiles/JoinToFiles/justBids/AsBid
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:33.982Z: Fusing consumer JoinToFiles/JoinToFiles.Debug into JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:34.022Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:34.096Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:34.130Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:34.174Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) into NexmarkUtils.GenerateSideInputData/MapElements/Map
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:34.216Z: Fusing consumer NexmarkUtils.GenerateSideInputData/MapElements/Map into NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:34.258Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:34.333Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:34.375Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:34.809Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Create
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:34.849Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:34.889Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Create
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:34.904Z: Starting 4 workers in us-central1-b...
Nov 15, 2018 4:21:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:35.188Z: Executing operation NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)+NexmarkUtils.GenerateSideInputData/MapElements/Map+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 15, 2018 4:21:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:21:46.639Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 15, 2018 4:22:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:22:16.442Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 15, 2018 4:22:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:22:16.482Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 15, 2018 4:22:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:22:16.839Z: Workers have started successfully.
Nov 15, 2018 4:22:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:22:16.887Z: Workers have started successfully.
Nov 15, 2018 4:22:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:22:34.732Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 15, 2018 4:22:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:22:41.690Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 15, 2018 4:22:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:22:41.805Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write
Nov 15, 2018 4:22:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:22:48.844Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Close
Nov 15, 2018 4:22:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:22:48.880Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Close
Nov 15, 2018 4:22:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:22:48.941Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey)
Nov 15, 2018 4:22:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:22:48.983Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize)
Nov 15, 2018 4:22:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:22:58.651Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections
Nov 15, 2018 4:22:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:22:59.186Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView
Nov 15, 2018 4:23:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:22:59.603Z: Executing operation JoinToFiles.ReadBounded+JoinToFiles/JoinToFiles.Monitor+JoinToFiles/JoinToFiles.Snoop+JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)+JoinToFiles/JoinToFiles/justBids/AsBid+JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles+JoinToFiles/JoinToFiles.Debug+JoinToFiles/JoinToFiles.Stamp+JoinToFiles.Format+JoinToFiles.DevNull
Nov 15, 2018 4:24:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:24:00.733Z: Cleaning up.
Nov 15, 2018 4:24:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T16:24:00.872Z: Stopping worker pool...
Nov 15, 2018 4:25:25 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-15T16:25:25.083Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-15T16:25:25.115Z RUNNING JoinToFiles
2018-11-15T16:25:26.480Z new perf {"snapshots":null,"runtimeSec":58.173,"numEvents":10000000,"eventsPerSec":171901.0537534595,"eventBytesPerSec":2.206461583552507E7,"numResults":9200000,"resultsPerSec":158148.96945318274,"resultBytesPerSec":5637182.111976346,"startupDelaySec":-143.115,"processingDelaySec":0.81,"shutdownDelaySec":84.942,"timeDilation":17.189245870077183,"errors":null,"jobId":null}
2018-11-15T16:25:41.549Z RUNNING JoinToFiles
2018-11-15T16:25:42.575Z no activity
2018-11-15T16:25:57.624Z RUNNING JoinToFiles
2018-11-15T16:25:58.590Z no activity
2018-11-15T16:26:13.627Z RUNNING JoinToFiles
2018-11-15T16:26:14.670Z no activity
2018-11-15T16:26:29.706Z RUNNING JoinToFiles
2018-11-15T16:26:30.910Z no activity
2018-11-15T16:26:45.962Z DONE JoinToFiles
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Daemon worker,5,main]) completed. Took 5 mins 31.096 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 36s
76 actionable tasks: 76 executed
Publishing build scan...
https://gradle.com/s/n5lexkbwwbeky
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1041
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1041/display/redirect?page=changes>
Changes:
[vaclav.plajt] [BEAM-6062] MetricsContainerStepMap#toString overwritten to show Beam
------------------------------------------
[...truncated 255.98 KB...]
INFO: Adding Query7/Query7.Monitor as step s2
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88169 bytes, hash e-xgp3wqxBFZ9jF6CEhLgQ> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-e-xgp3wqxBFZ9jF6CEhLgQ.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 15, 2018 12:46:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-15_04_46_43-7270627279516513033
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-15_04_46_43-7270627279516513033?project=apache-beam-testing
Nov 15, 2018 12:46:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-15_04_46_43-7270627279516513033
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-15T12:46:44.121Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:46.693Z: Checking permissions granted to controller Service Account.
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:50.241Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:50.717Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:50.822Z: Expanding GroupByKey operations into optimizable parts.
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:50.873Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.081Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.123Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.164Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.211Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.265Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.341Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.407Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.458Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.513Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.560Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.604Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.653Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.706Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.762Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.812Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.866Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.906Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:51.954Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:52.017Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:52.060Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:52.107Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:52.156Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:52.223Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:52.275Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:52.322Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:52.368Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:52.408Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:52.439Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:52.486Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:52.531Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:52.582Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:52.635Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:52.683Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:52.734Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:53.354Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:53.410Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:53.461Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:53.476Z: Starting 4 workers in us-central1-b...
Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:46:53.781Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 15, 2018 12:47:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:47:10.379Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 15, 2018 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:47:38.759Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 15, 2018 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:47:38.851Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 15, 2018 12:47:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:47:39.535Z: Workers have started successfully.
Nov 15, 2018 12:47:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:47:39.582Z: Workers have started successfully.
Nov 15, 2018 12:48:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-15T12:48:10.628Z: Autoscaling: Startup of the worker pool in zone us-central1-b reached 2 workers, but the goal was 4 workers. The service will retry. QUOTA_EXCEEDED: Quota 'CPUS' exceeded. Limit: 750.0 in region us-central1.
Nov 15, 2018 12:49:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:49:29.849Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 15, 2018 12:49:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:49:29.887Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 15, 2018 12:50:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:50:11.630Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 15, 2018 12:50:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:50:38.217Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 15, 2018 12:50:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:50:38.310Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 15, 2018 12:50:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:50:40.949Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 15, 2018 12:50:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:50:41.040Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 15, 2018 12:50:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:50:42.505Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 15, 2018 12:50:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:50:42.604Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 15, 2018 12:50:44 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-11-15T12:50:44.940Z Waiting for main pipeline to 'finish'
2018-11-15T12:50:44.981Z RUNNING Query7
Nov 15, 2018 12:50:45 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric result.elements, from namespace Query7.Results
Nov 15, 2018 12:50:45 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric result.bytes, from namespace Query7.Results
Nov 15, 2018 12:50:45 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getDistributionMetric
SEVERE: Failed to get distribution metric result.startTime for namespace Query7.Results
Nov 15, 2018 12:50:45 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getDistributionMetric
SEVERE: Failed to get distribution metric result.endTime for namespace Query7.Results
Nov 15, 2018 12:50:46 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getDistributionMetric
SEVERE: Failed to get distribution metric result.startTimestamp for namespace Query7.Results
Nov 15, 2018 12:50:46 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getDistributionMetric
SEVERE: Failed to get distribution metric result.endTimestamp for namespace Query7.Results
2018-11-15T12:50:46.266Z new perf {"snapshots":null,"runtimeSec":161.414,"numEvents":10000000,"eventsPerSec":61952.494826966686,"eventBytesPerSec":7952004.764146853,"numResults":-1,"resultsPerSec":-1.0,"resultBytesPerSec":-1.0,"startupDelaySec":-170.493,"processingDelaySec":-1.0,"shutdownDelaySec":-1.0,"timeDilation":-1.0,"errors":null,"jobId":null}
2018-11-15T12:51:01.325Z RUNNING Query7
2018-11-15T12:51:02.468Z new perf {"snapshots":null,"runtimeSec":186.494,"numEvents":10000000,"eventsPerSec":53621.028022349245,"eventBytesPerSec":6882606.931054082,"numResults":49,"resultsPerSec":0.2627430373095113,"resultBytesPerSec":26.483425740238292,"startupDelaySec":-170.493,"processingDelaySec":173.985,"shutdownDelaySec":0.329,"timeDilation":5.1103520756699945,"errors":null,"jobId":null}
2018-11-15T12:51:17.562Z RUNNING Query7
2018-11-15T12:51:18.920Z new perf {"snapshots":null,"runtimeSec":196.804,"numEvents":10000000,"eventsPerSec":50811.975366354345,"eventBytesPerSec":6522046.792748115,"numResults":100,"resultsPerSec":0.5081197536635434,"resultBytesPerSec":51.335338712627795,"startupDelaySec":-170.493,"processingDelaySec":173.985,"shutdownDelaySec":6.218,"timeDilation":4.996849657527286,"errors":null,"jobId":null}
2018-11-15T12:51:33.967Z RUNNING Query7
2018-11-15T12:51:35.325Z no activity
2018-11-15T12:51:50.406Z RUNNING Query7
2018-11-15T12:51:51.940Z no activity
2018-11-15T12:52:07.005Z RUNNING Query7
2018-11-15T12:52:08.272Z no activity
2018-11-15T12:52:23.332Z RUNNING Query7
2018-11-15T12:52:24.901Z no activity
2018-11-15T12:52:39.946Z RUNNING Query7
2018-11-15T12:52:41.255Z no activity
2018-11-15T12:52:56.301Z RUNNING Query7
2018-11-15T12:52:57.612Z no activity
2018-11-15T12:53:12.666Z RUNNING Query7
2018-11-15T12:53:13.746Z no activity
2018-11-15T12:53:28.779Z RUNNING Query7
2018-11-15T12:53:29.827Z no activity
2018-11-15T12:53:44.876Z RUNNING Query7
2018-11-15T12:53:45.986Z no activity
2018-11-15T12:54:01.106Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 4,5,main]) completed. Took 7 mins 42.919 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 11m 27s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/7ewisnl3jplvw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1040
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1040/display/redirect>
------------------------------------------
[...truncated 258.05 KB...]
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-JNUuGKDzwSjFNDFWItRl1w.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 15, 2018 12:03:59 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-JNUuGKDzwSjFNDFWItRl1w.jar
Nov 15, 2018 12:04:02 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 114 files cached, 29 files newly uploaded
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 15, 2018 12:04:03 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88173 bytes, hash 0oS64N8OvJl0oC4uC468uA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-0oS64N8OvJl0oC4uC468uA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-15_04_04_03-4354008152935297590
Nov 15, 2018 12:04:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-15_04_04_03-4354008152935297590?project=apache-beam-testing
Nov 15, 2018 12:04:04 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-15_04_04_03-4354008152935297590
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-15T12:04:03.923Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:06.764Z: Checking permissions granted to controller Service Account.
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:11.149Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:11.545Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:11.617Z: Expanding GroupByKey operations into optimizable parts.
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:11.652Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:11.809Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:11.844Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#25398a999f74762b
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:11.885Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:11.926Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:11.962Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:11.994Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.032Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.071Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.107Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.151Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.187Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.226Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.265Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.333Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.396Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.455Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.511Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.553Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.595Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.627Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.662Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.700Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.739Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.775Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.814Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.849Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.886Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.925Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.964Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:12.990Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:13.027Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:13.064Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:13.101Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:13.137Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:13.579Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:13.636Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:13.649Z: Starting 4 workers in us-central1-b...
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:13.671Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 15, 2018 12:04:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:13.934Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 15, 2018 12:04:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:04:30.249Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 15, 2018 12:05:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:05:01.738Z: Workers have started successfully.
Nov 15, 2018 12:05:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:05:04.625Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 15, 2018 12:05:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:05:04.659Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 15, 2018 12:05:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:05:10.618Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 15, 2018 12:05:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:05:32.916Z: Workers have started successfully.
Nov 15, 2018 12:06:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:06:24.898Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 15, 2018 12:06:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:06:24.985Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 15, 2018 12:06:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:06:25.955Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 15, 2018 12:06:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:06:26.042Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 15, 2018 12:06:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:06:28.714Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 15, 2018 12:06:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:06:28.835Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 15, 2018 12:06:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:06:32.411Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 15, 2018 12:06:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:06:32.631Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 15, 2018 12:07:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:06:59.637Z: Cleaning up.
Nov 15, 2018 12:07:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T12:06:59.812Z: Stopping worker pool...
Nov 15, 2018 12:08:04 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-15T12:08:04.809Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-15T12:08:04.848Z RUNNING Query7
2018-11-15T12:08:05.956Z new perf {"snapshots":null,"runtimeSec":102.776,"numEvents":10000000,"eventsPerSec":97298.98030668638,"eventBytesPerSec":1.2488955563555695E7,"numResults":100,"resultsPerSec":0.9729898030668639,"resultBytesPerSec":98.30115980384527,"startupDelaySec":-168.471,"processingDelaySec":77.847,"shutdownDelaySec":65.695,"timeDilation":9.56838172335954,"errors":null,"jobId":null}
2018-11-15T12:08:20.999Z RUNNING Query7
2018-11-15T12:08:21.991Z no activity
2018-11-15T12:08:37.032Z RUNNING Query7
2018-11-15T12:08:38.002Z no activity
2018-11-15T12:08:53.041Z RUNNING Query7
2018-11-15T12:08:54.088Z no activity
2018-11-15T12:09:09.125Z RUNNING Query7
2018-11-15T12:09:10.142Z no activity
2018-11-15T12:09:25.179Z RUNNING Query7
2018-11-15T12:09:26.357Z no activity
2018-11-15T12:09:41.392Z RUNNING Query7
2018-11-15T12:09:42.514Z no activity
2018-11-15T12:09:57.559Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 4,5,main]) completed. Took 6 mins 21.844 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 50s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/xuddgthvtwo7k
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1039
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1039/display/redirect?page=changes>
Changes:
[lukasz.gajowy] [BEAM-6011] Create builder for phrase triggered only jobs
[lukasz.gajowy] [BEAM-6011] Create builder for NEXMark Jobs and Suites
[lukasz.gajowy] [BEAM-6011] Add NEXMark jobs that can be phrase-triggered
------------------------------------------
[...truncated 231.05 KB...]
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-cmPxDgAM3IRt-FOpNb0bYg.jar
Nov 15, 2018 10:33:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-cmPxDgAM3IRt-FOpNb0bYg.jar
Nov 15, 2018 10:33:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-cmPxDgAM3IRt-FOpNb0bYg.jar
Nov 15, 2018 10:33:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-cmPxDgAM3IRt-FOpNb0bYg.jar
Nov 15, 2018 10:33:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-cmPxDgAM3IRt-FOpNb0bYg.jar
Nov 15, 2018 10:33:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-cmPxDgAM3IRt-FOpNb0bYg.jar
Nov 15, 2018 10:33:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-cmPxDgAM3IRt-FOpNb0bYg.jar
Nov 15, 2018 10:33:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-cmPxDgAM3IRt-FOpNb0bYg.jar
Nov 15, 2018 10:33:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-cmPxDgAM3IRt-FOpNb0bYg.jar
Nov 15, 2018 10:33:17 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 114 files cached, 29 files newly uploaded
Nov 15, 2018 10:33:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 15, 2018 10:33:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 15, 2018 10:33:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 15, 2018 10:33:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 15, 2018 10:33:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 15, 2018 10:33:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 15, 2018 10:33:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 15, 2018 10:33:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 15, 2018 10:33:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 15, 2018 10:33:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 15, 2018 10:33:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 15, 2018 10:33:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 15, 2018 10:33:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 15, 2018 10:33:18 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 15, 2018 10:33:18 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 15, 2018 10:33:18 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 15, 2018 10:33:18 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 15, 2018 10:33:18 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 15, 2018 10:33:18 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 15, 2018 10:33:18 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 15, 2018 10:33:18 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 15, 2018 10:33:18 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 15, 2018 10:33:18 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 15, 2018 10:33:18 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 15, 2018 10:33:18 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 15, 2018 10:33:18 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 15, 2018 10:33:18 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 15, 2018 10:33:18 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 15, 2018 10:33:18 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88162 bytes, hash -DxwMOple5W2wxKNhoUE2A> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline--DxwMOple5W2wxKNhoUE2A.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-15_02_33_18-16188152647911993940
Nov 15, 2018 10:33:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-15_02_33_18-16188152647911993940?project=apache-beam-testing
Nov 15, 2018 10:33:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-15_02_33_18-16188152647911993940
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-15T10:33:18.636Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:21.556Z: Checking permissions granted to controller Service Account.
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:25.224Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:25.716Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:25.804Z: Expanding GroupByKey operations into optimizable parts.
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:25.849Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:26.059Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:26.102Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:26.148Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:26.195Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:26.240Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:26.291Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:26.338Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:26.385Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:26.430Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:26.478Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:26.562Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:26.650Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:26.817Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:26.909Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.008Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.083Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.138Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.187Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.234Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.280Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.326Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.373Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.419Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.468Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.512Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.556Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.592Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.642Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.690Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.741Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.788Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.828Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.873Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:27.913Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:28.429Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:28.479Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:28.526Z: Starting 4 workers in us-central1-b...
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:28.526Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 15, 2018 10:33:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:28.871Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 15, 2018 10:33:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:33:47.930Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 15, 2018 10:34:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:34:22.163Z: Workers have started successfully.
Nov 15, 2018 10:34:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:34:22.654Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 15, 2018 10:34:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:34:22.711Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 15, 2018 10:34:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:34:28.157Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 15, 2018 10:34:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:34:53.390Z: Workers have started successfully.
Nov 15, 2018 10:35:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:35:40.741Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 15, 2018 10:35:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:35:40.862Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 15, 2018 10:35:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:35:42.485Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 15, 2018 10:35:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:35:42.588Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 15, 2018 10:35:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:35:44.052Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 15, 2018 10:35:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:35:44.135Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 15, 2018 10:35:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:35:47.616Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 15, 2018 10:35:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:35:47.837Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 15, 2018 10:36:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:36:14.499Z: Cleaning up.
Nov 15, 2018 10:36:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T10:36:14.608Z: Stopping worker pool...
Nov 15, 2018 10:37:19 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-15T10:37:19.545Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-15T10:37:19.580Z RUNNING Query7
2018-11-15T10:37:20.870Z new perf {"snapshots":null,"runtimeSec":98.13,"numEvents":10000000,"eventsPerSec":101905.6353816366,"eventBytesPerSec":1.3080249638234995E7,"numResults":100,"resultsPerSec":1.0190563538163662,"resultBytesPerSec":102.95526342606746,"startupDelaySec":-163.737,"processingDelaySec":73.566,"shutdownDelaySec":65.607,"timeDilation":10.021400183430144,"errors":null,"jobId":null}
2018-11-15T10:37:35.903Z RUNNING Query7
2018-11-15T10:37:37.321Z no activity
2018-11-15T10:37:52.357Z RUNNING Query7
2018-11-15T10:37:53.343Z no activity
2018-11-15T10:38:08.377Z RUNNING Query7
2018-11-15T10:38:09.456Z no activity
2018-11-15T10:38:24.517Z RUNNING Query7
2018-11-15T10:38:25.544Z no activity
2018-11-15T10:38:40.587Z RUNNING Query7
2018-11-15T10:38:41.622Z no activity
2018-11-15T10:38:56.681Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 3,5,main]) completed. Took 5 mins 47.446 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 19s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/iv4tk2fojd6lw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1038
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1038/display/redirect>
------------------------------------------
[...truncated 262.36 KB...]
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-KfizwJMY2Z3NUmKlH8LMGQ.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-KfizwJMY2Z3NUmKlH8LMGQ.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 15, 2018 6:03:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-KfizwJMY2Z3NUmKlH8LMGQ.jar
Nov 15, 2018 6:03:56 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-KfizwJMY2Z3NUmKlH8LMGQ.jar
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 114 files cached, 29 files newly uploaded
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 15, 2018 6:04:00 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88174 bytes, hash 8-2JIZ8mMH417Z6l--onqA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-8-2JIZ8mMH417Z6l--onqA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-14_22_04_01-17168636653099936394
Nov 15, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-14_22_04_01-17168636653099936394?project=apache-beam-testing
Nov 15, 2018 6:04:02 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-14_22_04_01-17168636653099936394
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-15T06:04:01.277Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:04.903Z: Checking permissions granted to controller Service Account.
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:11.491Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:12.013Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:12.109Z: Expanding GroupByKey operations into optimizable parts.
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:12.156Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:12.360Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:12.409Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:12.463Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:12.521Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:12.570Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:12.619Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:12.666Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:12.710Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:12.755Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:12.805Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:12.851Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:12.902Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:12.965Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.057Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.130Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.192Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.253Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.303Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.360Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.409Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.455Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.504Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.553Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.599Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.655Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.703Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.750Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.796Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.843Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.895Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.945Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:13.995Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:14.046Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:14.093Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:14.619Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:14.668Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:14.708Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:14.730Z: Starting 4 workers in us-central1-b...
Nov 15, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:15.302Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 15, 2018 6:04:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:04:34.284Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 15, 2018 6:05:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:05:02.201Z: Workers have started successfully.
Nov 15, 2018 6:05:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:05:13.369Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 15, 2018 6:05:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:05:33.053Z: Workers have started successfully.
Nov 15, 2018 6:06:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:06:24.654Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 15, 2018 6:06:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:06:24.759Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 15, 2018 6:06:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:06:26.470Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 15, 2018 6:06:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:06:26.575Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 15, 2018 6:06:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:06:28.515Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 15, 2018 6:06:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:06:28.596Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 15, 2018 6:06:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:06:32.009Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 15, 2018 6:06:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:06:32.227Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 15, 2018 6:06:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:06:58.680Z: Cleaning up.
Nov 15, 2018 6:06:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T06:06:58.789Z: Stopping worker pool...
Nov 15, 2018 6:08:02 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-11-15T06:08:02.202Z Waiting for main pipeline to 'finish'
2018-11-15T06:08:02.241Z RUNNING Query7
2018-11-15T06:08:03.414Z new perf {"snapshots":null,"runtimeSec":100.445,"numEvents":10000000,"eventsPerSec":99556.97147692768,"eventBytesPerSec":1.2778783383941462E7,"numResults":100,"resultsPerSec":0.9955697147692768,"resultBytesPerSec":100.58240828314004,"startupDelaySec":-165.575,"processingDelaySec":77.882,"shutdownDelaySec":65.131,"timeDilation":9.790432575041068,"errors":null,"jobId":null}
2018-11-15T06:08:18.456Z RUNNING Query7
2018-11-15T06:08:19.624Z no activity
2018-11-15T06:08:34.664Z RUNNING Query7
2018-11-15T06:08:35.912Z no activity
2018-11-15T06:08:50.949Z RUNNING Query7
2018-11-15T06:08:52.384Z no activity
2018-11-15T06:09:07.494Z RUNNING Query7
2018-11-15T06:09:08.955Z no activity
2018-11-15T06:09:23.996Z RUNNING Query7
2018-11-15T06:09:25.229Z no activity
2018-11-15T06:09:40.273Z RUNNING Query7
2018-11-15T06:09:41.361Z no activity
2018-11-15T06:09:56.426Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Daemon worker,5,main]) completed. Took 6 mins 19.73 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 47s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/4o5nkvnzqlfpe
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1037
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1037/display/redirect?page=changes>
Changes:
[lcwik] [BEAM-6067] Specify pipeline_coder_id property in non-Beam-standard
------------------------------------------
[...truncated 236.85 KB...]
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-e6SXQewxDkFMcQsxHMZHOA.jar
Nov 15, 2018 3:35:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-e6SXQewxDkFMcQsxHMZHOA.jar
Nov 15, 2018 3:35:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-e6SXQewxDkFMcQsxHMZHOA.jar
Nov 15, 2018 3:35:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-e6SXQewxDkFMcQsxHMZHOA.jar
Nov 15, 2018 3:35:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-e6SXQewxDkFMcQsxHMZHOA.jar
Nov 15, 2018 3:35:13 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-e6SXQewxDkFMcQsxHMZHOA.jar
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 114 files cached, 29 files newly uploaded
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 15, 2018 3:35:17 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88189 bytes, hash Hh0J5us-bS75UDrpSDCBAg> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-Hh0J5us-bS75UDrpSDCBAg.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 15, 2018 3:35:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-14_19_35_18-9211148488530276017
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-14_19_35_18-9211148488530276017?project=apache-beam-testing
Nov 15, 2018 3:35:19 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-14_19_35_18-9211148488530276017
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-15T03:35:18.275Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:22.110Z: Checking permissions granted to controller Service Account.
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:26.671Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:27.291Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:27.411Z: Expanding GroupByKey operations into optimizable parts.
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:27.454Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:27.654Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:27.706Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#5e4921c369c80b03
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:27.763Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:27.810Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:27.848Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:27.900Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:27.946Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:27.990Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.034Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.082Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.131Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.173Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.213Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.261Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.307Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.348Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.387Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.437Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.483Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.534Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.577Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.628Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.668Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.714Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.767Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.810Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.856Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.900Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.943Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:28.977Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:29.024Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:29.064Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:29.113Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:29.169Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:29.882Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:29.922Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:29.974Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:30.000Z: Starting 4 workers in us-central1-b...
Nov 15, 2018 3:35:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:30.313Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 15, 2018 3:35:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:35:48.162Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 15, 2018 3:36:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:36:17.685Z: Workers have started successfully.
Nov 15, 2018 3:36:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:36:17.807Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 15, 2018 3:36:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:36:17.858Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 15, 2018 3:36:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:36:20.538Z: Workers have started successfully.
Nov 15, 2018 3:36:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:36:29.658Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 15, 2018 3:36:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:36:29.718Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 15, 2018 3:36:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:36:41.499Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 15, 2018 3:37:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:37:39.127Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 15, 2018 3:37:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:37:39.268Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 15, 2018 3:37:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:37:41.185Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 15, 2018 3:37:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:37:41.352Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 15, 2018 3:37:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:37:42.751Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 15, 2018 3:37:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:37:42.868Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 15, 2018 3:37:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:37:45.611Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 15, 2018 3:37:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:37:45.847Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 15, 2018 3:38:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:38:11.191Z: Cleaning up.
Nov 15, 2018 3:38:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:38:11.306Z: Stopping worker pool...
Nov 15, 2018 3:39:19 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-15T03:39:19.149Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-15T03:39:19.184Z RUNNING Query7
2018-11-15T03:39:20.508Z new perf {"snapshots":null,"runtimeSec":97.25,"numEvents":10000000,"eventsPerSec":102827.76349614396,"eventBytesPerSec":1.3198610766066838E7,"numResults":100,"resultsPerSec":1.0282776349614395,"resultBytesPerSec":103.88688946015424,"startupDelaySec":-165.541,"processingDelaySec":74.288,"shutdownDelaySec":68.291,"timeDilation":10.112082262210796,"errors":null,"jobId":null}
2018-11-15T03:39:36.245Z RUNNING Query7
2018-11-15T03:39:37.994Z no activity
2018-11-15T03:39:53.030Z RUNNING Query7
2018-11-15T03:39:54.107Z no activity
2018-11-15T03:40:09.179Z RUNNING Query7
2018-11-15T03:40:10.612Z no activity
2018-11-15T03:40:25.665Z RUNNING Query7
2018-11-15T03:40:26.905Z no activity
2018-11-15T03:40:41.945Z RUNNING Query7
2018-11-15T03:40:43.367Z no activity
2018-11-15T03:40:58.399Z RUNNING Query7
2018-11-15T03:40:59.458Z no activity
2018-11-15T03:41:14.508Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 4,5,main]) completed. Took 6 mins 5.974 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_172/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 44s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/ut4qvep6ku3iq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1036
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1036/display/redirect?page=changes>
Changes:
[robinyqiu] Revert PR #6903 because it is breaking test error reporting
------------------------------------------
[...truncated 235.36 KB...]
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-ykGg69D06i06URRVGBDDfQ.jar
Nov 15, 2018 3:25:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-ykGg69D06i06URRVGBDDfQ.jar
Nov 15, 2018 3:25:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-ykGg69D06i06URRVGBDDfQ.jar
Nov 15, 2018 3:25:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-ykGg69D06i06URRVGBDDfQ.jar
Nov 15, 2018 3:25:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-ykGg69D06i06URRVGBDDfQ.jar
Nov 15, 2018 3:25:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-ykGg69D06i06URRVGBDDfQ.jar
Nov 15, 2018 3:25:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-ykGg69D06i06URRVGBDDfQ.jar
Nov 15, 2018 3:25:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-ykGg69D06i06URRVGBDDfQ.jar
Nov 15, 2018 3:25:29 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-ykGg69D06i06URRVGBDDfQ.jar
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 114 files cached, 29 files newly uploaded
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 15, 2018 3:25:33 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88183 bytes, hash elVRrXPaNha3-pdzFFBE8w> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-elVRrXPaNha3-pdzFFBE8w.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 15, 2018 3:25:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-14_19_25_34-11615654500640259902
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-14_19_25_34-11615654500640259902?project=apache-beam-testing
Nov 15, 2018 3:25:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-14_19_25_34-11615654500640259902
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-15T03:25:34.473Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:37.094Z: Checking permissions granted to controller Service Account.
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:43.631Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:44.297Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:44.370Z: Expanding GroupByKey operations into optimizable parts.
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:44.407Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:44.531Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:44.560Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:44.597Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:44.630Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:44.667Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:44.699Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:44.734Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:44.781Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:44.813Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:44.841Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:44.867Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:44.899Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:44.936Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:44.970Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.004Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.028Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.050Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.076Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.110Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.145Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.181Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.215Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.254Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.286Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.325Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.360Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.393Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.429Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.468Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.500Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.528Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.558Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.581Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.612Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:45.980Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:46.016Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:46.044Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:46.044Z: Starting 4 workers in us-central1-b...
Nov 15, 2018 3:25:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:46.414Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 15, 2018 3:25:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:25:57.333Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 15, 2018 3:26:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:26:25.499Z: Workers have started successfully.
Nov 15, 2018 3:26:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:26:32.579Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 15, 2018 3:26:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:26:32.617Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 15, 2018 3:26:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:26:38.208Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 15, 2018 3:26:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:26:56.691Z: Workers have started successfully.
Nov 15, 2018 3:27:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:27:44.681Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 15, 2018 3:27:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:27:44.765Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 15, 2018 3:27:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:27:45.879Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 15, 2018 3:27:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:27:45.962Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 15, 2018 3:27:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:27:46.507Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 15, 2018 3:27:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:27:46.601Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 15, 2018 3:27:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:27:50.080Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 15, 2018 3:27:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:27:50.288Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 15, 2018 3:28:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:28:18.896Z: Cleaning up.
Nov 15, 2018 3:28:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T03:28:18.984Z: Stopping worker pool...
Nov 15, 2018 3:29:35 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-15T03:29:35.373Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-15T03:29:35.412Z RUNNING Query7
2018-11-15T03:29:36.790Z new perf {"snapshots":null,"runtimeSec":97.625,"numEvents":10000000,"eventsPerSec":102432.77848911652,"eventBytesPerSec":1.3147911877080666E7,"numResults":100,"resultsPerSec":1.0243277848911652,"resultBytesPerSec":103.48783610755441,"startupDelaySec":-175.272,"processingDelaySec":71.907,"shutdownDelaySec":77.647,"timeDilation":10.073239436619717,"errors":null,"jobId":null}
2018-11-15T03:29:51.833Z RUNNING Query7
2018-11-15T03:29:53.019Z no activity
2018-11-15T03:30:08.058Z RUNNING Query7
2018-11-15T03:30:09.135Z no activity
2018-11-15T03:30:24.253Z RUNNING Query7
2018-11-15T03:30:25.527Z no activity
2018-11-15T03:30:40.639Z RUNNING Query7
2018-11-15T03:30:41.797Z no activity
2018-11-15T03:30:56.860Z RUNNING Query7
2018-11-15T03:30:57.870Z no activity
2018-11-15T03:31:12.924Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 2,5,main]) completed. Took 5 mins 47.232 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 56s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/rxpsuwj5wodkm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1035
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1035/display/redirect?page=changes>
Changes:
[adrianwit] [BEAM-5729] added database/sql based reader and writer
------------------------------------------
[...truncated 242.11 KB...]
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 15, 2018 12:29:32 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-54mVu1Dg77tix4KuPI8kng.jar
Nov 15, 2018 12:29:35 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 114 files cached, 29 files newly uploaded
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 15, 2018 12:29:36 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88173 bytes, hash UuoV5Vmoiol4enZAUSeiZw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-UuoV5Vmoiol4enZAUSeiZw.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 15, 2018 12:29:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-14_16_29_36-11715671957284695222
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-14_16_29_36-11715671957284695222?project=apache-beam-testing
Nov 15, 2018 12:29:37 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-14_16_29_36-11715671957284695222
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-15T00:29:36.917Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:39.461Z: Checking permissions granted to controller Service Account.
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:43.414Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:43.974Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.046Z: Expanding GroupByKey operations into optimizable parts.
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.080Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.235Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.275Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.305Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.348Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.385Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.420Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.452Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.485Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.517Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.550Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.587Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.620Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.656Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.692Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.730Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.766Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.802Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.839Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.876Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.911Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.946Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:44.980Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:45.015Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:45.051Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:45.086Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:45.121Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:45.154Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:45.192Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:45.226Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:45.263Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:45.300Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:45.336Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:45.370Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:45.407Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:45.796Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:45.825Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:45.861Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:45.871Z: Starting 4 workers in us-central1-b...
Nov 15, 2018 12:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:46.088Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 15, 2018 12:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:29:56.470Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 15, 2018 12:30:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:30:33.162Z: Workers have started successfully.
Nov 15, 2018 12:30:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:30:37.595Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 15, 2018 12:30:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:30:37.634Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 15, 2018 12:30:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:30:43.098Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 15, 2018 12:30:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:30:43.140Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 15, 2018 12:30:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:30:54.761Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 15, 2018 12:30:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:30:57.362Z: Workers have started successfully.
Nov 15, 2018 12:31:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:31:51.514Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 15, 2018 12:31:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:31:51.584Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 15, 2018 12:31:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:31:52.299Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 15, 2018 12:31:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:31:52.376Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 15, 2018 12:31:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:31:54.238Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 15, 2018 12:31:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:31:54.327Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 15, 2018 12:31:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:31:57.637Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 15, 2018 12:31:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:31:57.903Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 15, 2018 12:32:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:32:23.669Z: Cleaning up.
Nov 15, 2018 12:32:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:32:23.956Z: Stopping worker pool...
Nov 15, 2018 12:33:37 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-15T00:33:37.548Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-15T00:33:37.585Z RUNNING Query7
2018-11-15T00:33:38.793Z new perf {"snapshots":null,"runtimeSec":96.169,"numEvents":10000000,"eventsPerSec":103983.61218272001,"eventBytesPerSec":1.3346971446100095E7,"numResults":100,"resultsPerSec":1.0398361218272,"resultBytesPerSec":105.05464338820202,"startupDelaySec":-170.778,"processingDelaySec":73.827,"shutdownDelaySec":74.609,"timeDilation":10.225748422048685,"errors":null,"jobId":null}
2018-11-15T00:33:53.857Z RUNNING Query7
2018-11-15T00:33:54.779Z no activity
2018-11-15T00:34:09.826Z RUNNING Query7
2018-11-15T00:34:10.989Z no activity
2018-11-15T00:34:26.024Z RUNNING Query7
2018-11-15T00:34:26.958Z no activity
2018-11-15T00:34:41.993Z RUNNING Query7
2018-11-15T00:34:42.950Z no activity
2018-11-15T00:34:58.015Z RUNNING Query7
2018-11-15T00:34:59.120Z no activity
2018-11-15T00:35:14.164Z RUNNING Query7
2018-11-15T00:35:15.103Z no activity
2018-11-15T00:35:30.191Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 5,5,main]) completed. Took 6 mins 10.616 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 53s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/2yjy6or7riqwc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1034
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1034/display/redirect>
------------------------------------------
[...truncated 246.32 KB...]
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Kohn-9hft0FIQ03bMTyfSg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 15, 2018 12:04:30 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Kohn-9hft0FIQ03bMTyfSg.jar
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 114 files cached, 29 files newly uploaded
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 15, 2018 12:04:34 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88170 bytes, hash NY-QRGsmxxBe7xDUXaTsmw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-NY-QRGsmxxBe7xDUXaTsmw.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 15, 2018 12:04:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-14_16_04_35-3723071061220915856
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-14_16_04_35-3723071061220915856?project=apache-beam-testing
Nov 15, 2018 12:04:36 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-14_16_04_35-3723071061220915856
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-15T00:04:35.292Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:38.221Z: Checking permissions granted to controller Service Account.
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:47.671Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:48.326Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:48.409Z: Expanding GroupByKey operations into optimizable parts.
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:48.449Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:48.663Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:48.705Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#37bc5e2e3faec20a
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:48.744Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:48.786Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:48.833Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:48.869Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:48.917Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:48.980Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.029Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.082Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.143Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.228Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.309Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.378Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.457Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.508Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.568Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.608Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.650Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.694Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.722Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.761Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.805Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.853Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.898Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.934Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:49.981Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:50.029Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:50.062Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:50.109Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:50.149Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:50.190Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:50.227Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:50.273Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:50.748Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:50.787Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:50.830Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:50.841Z: Starting 4 workers in us-central1-b...
Nov 15, 2018 12:04:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:04:51.091Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 15, 2018 12:05:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:05:09.980Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 15, 2018 12:05:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:05:41.489Z: Workers have started successfully.
Nov 15, 2018 12:05:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:05:56.520Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 15, 2018 12:05:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:05:56.563Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 15, 2018 12:06:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:06:02.685Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 15, 2018 12:06:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:06:07.140Z: Workers have started successfully.
Nov 15, 2018 12:07:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:07:07.210Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 15, 2018 12:07:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:07:07.292Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 15, 2018 12:07:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:07:08.715Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 15, 2018 12:07:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:07:08.834Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 15, 2018 12:07:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:07:10.337Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 15, 2018 12:07:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:07:10.412Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 15, 2018 12:07:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:07:13.560Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 15, 2018 12:07:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:07:13.777Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 15, 2018 12:07:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:07:36.806Z: Cleaning up.
Nov 15, 2018 12:07:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-15T00:07:36.908Z: Stopping worker pool...
Nov 15, 2018 12:08:36 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-15T00:08:36.375Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-15T00:08:36.413Z RUNNING Query7
2018-11-15T00:08:37.695Z new perf {"snapshots":null,"runtimeSec":93.525,"numEvents":10000000,"eventsPerSec":106923.28254477412,"eventBytesPerSec":1.3724297214648489E7,"numResults":100,"resultsPerSec":1.0692328254477412,"resultBytesPerSec":108.02459235498529,"startupDelaySec":-153.997,"processingDelaySec":73.622,"shutdownDelaySec":60.472,"timeDilation":10.514835605453086,"errors":null,"jobId":null}
2018-11-15T00:08:52.736Z RUNNING Query7
2018-11-15T00:08:53.918Z no activity
2018-11-15T00:09:08.955Z RUNNING Query7
2018-11-15T00:09:10.085Z no activity
2018-11-15T00:09:25.128Z RUNNING Query7
2018-11-15T00:09:26.166Z no activity
2018-11-15T00:09:41.207Z RUNNING Query7
2018-11-15T00:09:42.254Z no activity
2018-11-15T00:09:57.306Z RUNNING Query7
2018-11-15T00:09:58.394Z no activity
2018-11-15T00:10:13.439Z RUNNING Query7
2018-11-15T00:10:14.467Z no activity
2018-11-15T00:10:29.559Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 7,5,main]) completed. Took 6 mins 17.239 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 22s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/rtfztkzvjmaic
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1033
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1033/display/redirect?page=changes>
Changes:
[altay] fix lint error un userstate.py
------------------------------------------
[...truncated 234.39 KB...]
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-fi_2dXL68r0FmSEBpxo71g.jar
Nov 14, 2018 8:13:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-fi_2dXL68r0FmSEBpxo71g.jar
Nov 14, 2018 8:13:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-fi_2dXL68r0FmSEBpxo71g.jar
Nov 14, 2018 8:13:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-fi_2dXL68r0FmSEBpxo71g.jar
Nov 14, 2018 8:13:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-fi_2dXL68r0FmSEBpxo71g.jar
Nov 14, 2018 8:13:26 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 114 files cached, 29 files newly uploaded
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 14, 2018 8:13:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88175 bytes, hash X208PKcbzgX_thyRhYvQFQ> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-X208PKcbzgX_thyRhYvQFQ.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 14, 2018 8:13:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-14_12_13_27-10583958177698885650
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-14_12_13_27-10583958177698885650?project=apache-beam-testing
Nov 14, 2018 8:13:28 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-14_12_13_27-10583958177698885650
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-14T20:13:27.843Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:30.784Z: Checking permissions granted to controller Service Account.
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:35.076Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:35.725Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:35.825Z: Expanding GroupByKey operations into optimizable parts.
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:35.878Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.054Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.089Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.138Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.188Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.232Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.266Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.321Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.362Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.409Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.455Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.491Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.542Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.582Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.638Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.762Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.829Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.875Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.918Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:36.957Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:37.007Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:37.050Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:37.103Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:37.155Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:37.198Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:37.241Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:37.289Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:37.329Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:37.367Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:37.420Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:37.466Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:37.515Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:37.566Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:37.617Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:37.655Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:38.107Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:38.151Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:38.183Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:38.183Z: Starting 4 workers in us-central1-b...
Nov 14, 2018 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:38.467Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 14, 2018 8:13:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:13:48.323Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 14, 2018 8:14:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:14:23.116Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 14, 2018 8:14:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:14:23.154Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 14, 2018 8:14:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:14:28.774Z: Workers have started successfully.
Nov 14, 2018 8:14:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:14:28.816Z: Workers have started successfully.
Nov 14, 2018 8:14:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:14:41.692Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 14, 2018 8:14:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:14:41.730Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 14, 2018 8:14:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:14:47.635Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 14, 2018 8:15:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:15:54.429Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 14, 2018 8:15:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:15:54.518Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 14, 2018 8:15:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:15:55.909Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 14, 2018 8:15:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:15:56.022Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 14, 2018 8:15:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:15:56.938Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 14, 2018 8:15:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:15:57.052Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 14, 2018 8:16:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:16:00.784Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 14, 2018 8:16:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:16:00.992Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 14, 2018 8:16:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:16:30.777Z: Cleaning up.
Nov 14, 2018 8:16:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:16:30.909Z: Stopping worker pool...
Nov 14, 2018 8:17:28 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-14T20:17:28.770Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-14T20:17:28.821Z RUNNING Query7
2018-11-14T20:17:29.982Z new perf {"snapshots":null,"runtimeSec":107.924,"numEvents":10000000,"eventsPerSec":92657.79622697453,"eventBytesPerSec":1.1893229467032356E7,"numResults":100,"resultsPerSec":0.9265779622697453,"resultBytesPerSec":93.61217152811237,"startupDelaySec":-166.269,"processingDelaySec":80.614,"shutdownDelaySec":58.345,"timeDilation":9.111967680960674,"errors":null,"jobId":null}
2018-11-14T20:17:45.040Z RUNNING Query7
2018-11-14T20:17:46.273Z no activity
2018-11-14T20:18:01.306Z RUNNING Query7
2018-11-14T20:18:02.542Z no activity
2018-11-14T20:18:17.579Z RUNNING Query7
2018-11-14T20:18:18.681Z no activity
2018-11-14T20:18:33.728Z RUNNING Query7
2018-11-14T20:18:34.875Z no activity
2018-11-14T20:18:49.918Z RUNNING Query7
2018-11-14T20:18:50.981Z no activity
2018-11-14T20:19:06.067Z RUNNING Query7
2018-11-14T20:19:07.180Z no activity
2018-11-14T20:19:22.230Z RUNNING Query7
2018-11-14T20:19:23.261Z no activity
2018-11-14T20:19:38.309Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 10,5,main]) completed. Took 6 mins 21.178 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 11s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/te5x5xsyakmu4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1032
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1032/display/redirect?page=changes>
Changes:
[altay] Convert Top combiner to a full PTransform
------------------------------------------
[...truncated 345.28 KB...]
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Snoop as step s5
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) as step s6
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s7
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) as step s8
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize as step s9
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) as step s10
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys as step s11
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) as step s12
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections as step s13
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView as step s14
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) as step s15
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/AsBid as step s16
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles as step s17
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Debug as step s18
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Stamp as step s19
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.Format as step s20
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.DevNull as step s21
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 14, 2018 8:03:39 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <58911 bytes, hash UMgQDEHpel4-QGk6xajYGQ> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-UMgQDEHpel4-QGk6xajYGQ.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 14, 2018 8:03:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-14_12_03_39-15367662654408494005
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-14_12_03_39-15367662654408494005?project=apache-beam-testing
Nov 14, 2018 8:03:40 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-14_12_03_39-15367662654408494005
Nov 14, 2018 8:03:42 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-mxOjcYQfrzMYL2s8cJL9RA.jar
Nov 14, 2018 8:03:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-mxOjcYQfrzMYL2s8cJL9RA.jar
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 122 files cached, 21 files newly uploaded
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 14, 2018 8:03:48 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88179 bytes, hash j8BIt_io-fAmSBVfnlwByg> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-j8BIt_io-fAmSBVfnlwByg.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-14T20:03:39.900Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:42.672Z: Checking permissions granted to controller Service Account.
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:47.732Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:48.324Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:48.488Z: Expanding GroupByKey operations into optimizable parts.
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:48.531Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:48.628Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:48.663Z: Fusing consumer JoinToFiles/JoinToFiles.Monitor into JoinToFiles.ReadBounded
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:48.713Z: Fusing consumer JoinToFiles/JoinToFiles.Snoop into JoinToFiles/JoinToFiles.Monitor
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:48.749Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) into JoinToFiles/JoinToFiles.Snoop
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:48.793Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/AsBid into JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:48.833Z: Fusing consumer JoinToFiles.Format into JoinToFiles/JoinToFiles.Stamp
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:48.877Z: Fusing consumer JoinToFiles.DevNull into JoinToFiles.Format
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:48.927Z: Fusing consumer JoinToFiles/JoinToFiles.Stamp into JoinToFiles/JoinToFiles.Debug
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:48.970Z: Fusing consumer JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles into JoinToFiles/JoinToFiles/justBids/AsBid
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:49.025Z: Fusing consumer JoinToFiles/JoinToFiles.Debug into JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:49.077Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:49.118Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:49.159Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:49.201Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) into NexmarkUtils.GenerateSideInputData/MapElements/Map
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:49.241Z: Fusing consumer NexmarkUtils.GenerateSideInputData/MapElements/Map into NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:49.281Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:49.324Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:49.360Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:49.836Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Create
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:49.880Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:49.926Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Create
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:49.937Z: Starting 4 workers in us-central1-b...
Nov 14, 2018 8:03:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:03:50.229Z: Executing operation NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)+NexmarkUtils.GenerateSideInputData/MapElements/Map+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 14, 2018 8:04:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:04:14.638Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 14, 2018 8:04:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:04:43.651Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 14, 2018 8:04:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:04:43.687Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 14, 2018 8:04:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:04:44.684Z: Workers have started successfully.
Nov 14, 2018 8:04:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:04:44.718Z: Workers have started successfully.
Nov 14, 2018 8:04:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:04:49.700Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 14, 2018 8:04:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:04:49.741Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 14, 2018 8:04:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:04:55.967Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 14, 2018 8:05:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:05:11.783Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 14, 2018 8:05:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:05:11.896Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write
Nov 14, 2018 8:05:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:05:19.532Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Close
Nov 14, 2018 8:05:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:05:19.576Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Close
Nov 14, 2018 8:05:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:05:19.640Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey)
Nov 14, 2018 8:05:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:05:19.692Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize)
Nov 14, 2018 8:05:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:05:28.210Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections
Nov 14, 2018 8:05:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:05:28.444Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView
Nov 14, 2018 8:05:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:05:28.650Z: Executing operation JoinToFiles.ReadBounded+JoinToFiles/JoinToFiles.Monitor+JoinToFiles/JoinToFiles.Snoop+JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)+JoinToFiles/JoinToFiles/justBids/AsBid+JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles+JoinToFiles/JoinToFiles.Debug+JoinToFiles/JoinToFiles.Stamp+JoinToFiles.Format+JoinToFiles.DevNull
Nov 14, 2018 8:06:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:06:27.265Z: Cleaning up.
Nov 14, 2018 8:06:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T20:06:27.365Z: Stopping worker pool...
Nov 14, 2018 8:07:40 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-11-14T20:07:40.803Z Waiting for main pipeline to 'finish'
2018-11-14T20:07:40.858Z RUNNING JoinToFiles
2018-11-14T20:07:41.998Z new perf {"snapshots":null,"runtimeSec":57.175,"numEvents":10000000,"eventsPerSec":174901.61783996504,"eventBytesPerSec":2.244975770878881E7,"numResults":9200000,"resultsPerSec":160909.48841276782,"resultBytesPerSec":5735580.148666375,"startupDelaySec":-131.01,"processingDelaySec":0.936,"shutdownDelaySec":73.835,"timeDilation":17.489287275907305,"errors":null,"jobId":null}
2018-11-14T20:07:57.035Z RUNNING JoinToFiles
2018-11-14T20:07:58.081Z no activity
2018-11-14T20:08:13.150Z RUNNING JoinToFiles
2018-11-14T20:08:14.186Z no activity
2018-11-14T20:08:29.243Z RUNNING JoinToFiles
2018-11-14T20:08:30.402Z no activity
2018-11-14T20:08:45.465Z RUNNING JoinToFiles
2018-11-14T20:08:46.519Z no activity
2018-11-14T20:09:01.579Z RUNNING JoinToFiles
2018-11-14T20:09:02.765Z no activity
2018-11-14T20:09:17.822Z DONE JoinToFiles
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 4,5,main]) completed. Took 6 mins 0.281 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 1s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/5tkow7lmzqrbu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1031
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1031/display/redirect?page=changes>
Changes:
[scott] Roll-forwawrd "Merge pull request #6979: Add helper task to print
[scott] Share docker image name along with shared task
------------------------------------------
[...truncated 225.33 KB...]
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/testing/nexmark/build/libs/beam-sdks-java-nexmark-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-nexmark-2.9.0-SNAPSHOT-Oo8yPft4afGHuUtDa2cSag.jar
Nov 14, 2018 7:49:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-io-google-cloud-platform-2.9.0-SNAPSHOT-08fkZxBCbhrZKnda1CysaQ.jar
Nov 14, 2018 7:49:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-extensions-join-library-2.9.0-SNAPSHOT-8uant0oTtkHUou0BTAuzQw.jar
Nov 14, 2018 7:49:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-extensions-sql-2.9.0-SNAPSHOT-1tSB0ETj22H6DiVjz5Noxw.jar
Nov 14, 2018 7:49:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-extensions-protobuf-2.9.0-SNAPSHOT-IMRidWgQ-Xn6Um3eBvVRqw.jar
Nov 14, 2018 7:49:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.9.0-SNAPSHOT-yB-16Mv13dDWZClX99zDPQ.jar
Nov 14, 2018 7:49:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-test-utils-2.9.0-SNAPSHOT-B0ZvL5qq6hVukekwunlczw.jar
Nov 14, 2018 7:49:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/io/kafka/build/libs/beam-sdks-java-io-kafka-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-io-kafka-2.9.0-SNAPSHOT-TexPdsf-3_MSHcyKmhfDFQ.jar
Nov 14, 2018 7:49:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-sdks-java-core-2.9.0-SNAPSHOT-JVva4Y2DcC98HPkb8V2zSA.jar
Nov 14, 2018 7:49:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-direct-java-2.9.0-SNAPSHOT-VUvL2X6F1U2zXVzlafbzZQ.jar
Nov 14, 2018 7:49:31 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-core-construction-java-2.9.0-SNAPSHOT-r1ldP4EQmqY10-jb7eCgSA.jar
Nov 14, 2018 7:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/model/job-management/build/libs/beam-model-job-management-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-model-job-management-2.9.0-SNAPSHOT-JxgV0nIfThPos4fSt_xSHQ.jar
Nov 14, 2018 7:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5vc4o8RoAf4tUitNpyU4rQ.jar
Nov 14, 2018 7:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5vc4o8RoAf4tUitNpyU4rQ.jar
Nov 14, 2018 7:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5vc4o8RoAf4tUitNpyU4rQ.jar
Nov 14, 2018 7:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5vc4o8RoAf4tUitNpyU4rQ.jar
Nov 14, 2018 7:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5vc4o8RoAf4tUitNpyU4rQ.jar
Nov 14, 2018 7:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5vc4o8RoAf4tUitNpyU4rQ.jar
Nov 14, 2018 7:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5vc4o8RoAf4tUitNpyU4rQ.jar
Nov 14, 2018 7:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-model-pipeline-2.9.0-SNAPSHOT-86AtKDAzD7xkAFn9dr-B_w.jar
Nov 14, 2018 7:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5vc4o8RoAf4tUitNpyU4rQ.jar
Nov 14, 2018 7:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/vendor/grpc-v1_13_1/build/libs/beam-vendor-grpc-v1_13_1-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-vendor-grpc-v1_13_1-2.9.0-SNAPSHOT-K-c06lbHDpz0DhC6yYF22Q.jar
Nov 14, 2018 7:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5vc4o8RoAf4tUitNpyU4rQ.jar
Nov 14, 2018 7:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5vc4o8RoAf4tUitNpyU4rQ.jar
Nov 14, 2018 7:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5vc4o8RoAf4tUitNpyU4rQ.jar
Nov 14, 2018 7:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5vc4o8RoAf4tUitNpyU4rQ.jar
Nov 14, 2018 7:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5vc4o8RoAf4tUitNpyU4rQ.jar
Nov 14, 2018 7:49:32 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-5vc4o8RoAf4tUitNpyU4rQ.jar
Nov 14, 2018 7:49:36 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 114 files cached, 29 files newly uploaded
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.ReadBounded as step s1
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource) as step s2
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding NexmarkUtils.GenerateSideInputData/MapElements/Map as step s3
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Monitor as step s4
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Snoop as step s5
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) as step s6
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s7
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) as step s8
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize as step s9
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) as step s10
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys as step s11
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) as step s12
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections as step s13
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView as step s14
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) as step s15
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/justBids/AsBid as step s16
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles as step s17
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Debug as step s18
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles/JoinToFiles.Stamp as step s19
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.Format as step s20
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding JoinToFiles.DevNull as step s21
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 14, 2018 7:49:37 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <58919 bytes, hash YLHiEEPVLeg23FkpHeDx3w> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-YLHiEEPVLeg23FkpHeDx3w.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 14, 2018 7:49:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-14_11_49_37-1054776469072204228
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-14_11_49_37-1054776469072204228?project=apache-beam-testing
Nov 14, 2018 7:49:38 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-14_11_49_37-1054776469072204228
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-14T19:49:38.072Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:41.226Z: Checking permissions granted to controller Service Account.
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:45.841Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:46.490Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:46.636Z: Expanding GroupByKey operations into optimizable parts.
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:46.679Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:46.769Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:46.811Z: Fusing consumer JoinToFiles/JoinToFiles.Monitor into JoinToFiles.ReadBounded
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:46.854Z: Fusing consumer JoinToFiles/JoinToFiles.Snoop into JoinToFiles/JoinToFiles.Monitor
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:46.894Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous) into JoinToFiles/JoinToFiles.Snoop
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:46.933Z: Fusing consumer JoinToFiles/JoinToFiles/justBids/AsBid into JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:46.968Z: Fusing consumer JoinToFiles.Format into JoinToFiles/JoinToFiles.Stamp
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:47.019Z: Fusing consumer JoinToFiles.DevNull into JoinToFiles.Format
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:47.054Z: Fusing consumer JoinToFiles/JoinToFiles.Stamp into JoinToFiles/JoinToFiles.Debug
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:47.101Z: Fusing consumer JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles into JoinToFiles/JoinToFiles/justBids/AsBid
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:47.138Z: Fusing consumer JoinToFiles/JoinToFiles.Debug into JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:47.176Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:47.216Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:47.256Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:47.290Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) into NexmarkUtils.GenerateSideInputData/MapElements/Map
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:47.322Z: Fusing consumer NexmarkUtils.GenerateSideInputData/MapElements/Map into NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:47.365Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike) into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:47.400Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write into JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:47.423Z: Fusing consumer JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:47.903Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Create
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:47.943Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:47.984Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Create
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:47.995Z: Starting 4 workers in us-central1-b...
Nov 14, 2018 7:49:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:49:48.301Z: Executing operation NexmarkUtils.GenerateSideInputData/GenerateSequence/Read(BoundedCountingSource)+NexmarkUtils.GenerateSideInputData/MapElements/Map+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 14, 2018 7:50:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:50:02.070Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 14, 2018 7:50:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:50:48.465Z: Workers have started successfully.
Nov 14, 2018 7:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:50:54.196Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 14, 2018 7:50:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:50:54.296Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 14, 2018 7:51:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:51:19.710Z: Workers have started successfully.
Nov 14, 2018 7:51:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:51:24.464Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 14, 2018 7:51:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:51:24.568Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+JoinToFiles/JoinToFiles/View.AsMap/ParMultiDo(ToIsmRecordForMapLike)+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Write+JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Write
Nov 14, 2018 7:51:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:51:34.843Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 14, 2018 7:51:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:51:34.882Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 14, 2018 7:51:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:51:44.067Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Close
Nov 14, 2018 7:51:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:51:44.120Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Close
Nov 14, 2018 7:51:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:51:44.163Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForKeys/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForKey)
Nov 14, 2018 7:51:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:51:44.210Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/GBKaSVForSize/Read+JoinToFiles/JoinToFiles/View.AsMap/ParDo(ToIsmMetadataRecordForSize)
Nov 14, 2018 7:51:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:51:51.746Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 14, 2018 7:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:51:53.787Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/Flatten.PCollections
Nov 14, 2018 7:51:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:51:54.283Z: Executing operation JoinToFiles/JoinToFiles/View.AsMap/CreateDataflowView
Nov 14, 2018 7:51:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:51:55.066Z: Executing operation JoinToFiles.ReadBounded+JoinToFiles/JoinToFiles.Monitor+JoinToFiles/JoinToFiles.Snoop+JoinToFiles/JoinToFiles/justBids/IsBid/ParDo(Anonymous)+JoinToFiles/JoinToFiles/justBids/AsBid+JoinToFiles/JoinToFiles/JoinToFiles.JoinToFiles+JoinToFiles/JoinToFiles.Debug+JoinToFiles/JoinToFiles.Stamp+JoinToFiles.Format+JoinToFiles.DevNull
Nov 14, 2018 7:53:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:53:00.512Z: Cleaning up.
Nov 14, 2018 7:53:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T19:53:00.632Z: Stopping worker pool...
Nov 14, 2018 7:53:38 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned. State value RUNNING
2018-11-14T19:53:38.886Z Waiting for main pipeline to 'finish'
2018-11-14T19:53:38.931Z RUNNING JoinToFiles
2018-11-14T19:53:40.074Z new perf {"snapshots":null,"runtimeSec":62.946,"numEvents":10000000,"eventsPerSec":158866.32987004734,"eventBytesPerSec":2.0391524433641534E7,"numResults":9200000,"resultsPerSec":146157.02348044355,"resultBytesPerSec":5209732.071934674,"startupDelaySec":-101.547,"processingDelaySec":0.857,"shutdownDelaySec":38.601,"timeDilation":15.885838655355386,"errors":null,"jobId":null}
2018-11-14T19:53:55.114Z RUNNING JoinToFiles
2018-11-14T19:53:56.302Z no activity
2018-11-14T19:54:15.237Z RUNNING JoinToFiles
2018-11-14T19:54:37.109Z no activity
2018-11-14T19:54:52.170Z RUNNING JoinToFiles
2018-11-14T19:54:53.667Z no activity
2018-11-14T19:55:08.713Z RUNNING JoinToFiles
2018-11-14T19:55:09.830Z no activity
2018-11-14T19:55:24.867Z RUNNING JoinToFiles
2018-11-14T19:55:25.894Z no activity
2018-11-14T19:55:40.947Z DONE JoinToFiles
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 6,5,main]) completed. Took 6 mins 12.955 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 56s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/hejb5tauzdqoq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1030
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1030/display/redirect?page=changes>
Changes:
[mxm] ParDoTest: Check expected errors inline instead of after pipeline
[mxm] Add output collection id to timer specs.
[mxm] Move doFn wrapper initialization to open()
[mxm] [BEAM-6009] Add note about default mode for PortableValidatesRunner
[mxm] [BEAM-4681] Add support for portable timers in Flink streaming mode
------------------------------------------
[...truncated 246.25 KB...]
"errors" : [ {
"domain" : "usageLimits",
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Rc_wgLdGSia4wa6op9S8Fw.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
"reason" : "rateLimitExceeded"
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Rc_wgLdGSia4wa6op9S8Fw.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 14, 2018 6:56:15 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Rc_wgLdGSia4wa6op9S8Fw.jar
Nov 14, 2018 6:56:19 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 114 files cached, 29 files newly uploaded
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 14, 2018 6:56:20 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88183 bytes, hash -JVr8613S_T0fFYv5s6C2Q> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline--JVr8613S_T0fFYv5s6C2Q.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 14, 2018 6:56:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-14_10_56_20-455537278409340018
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-14_10_56_20-455537278409340018?project=apache-beam-testing
Nov 14, 2018 6:56:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-14_10_56_20-455537278409340018
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-14T18:56:20.810Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:23.729Z: Checking permissions granted to controller Service Account.
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:29.118Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:29.676Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:29.777Z: Expanding GroupByKey operations into optimizable parts.
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:29.834Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.067Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.110Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#6044f02ac45a6f71
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.166Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.221Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.266Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.311Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.354Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.406Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.459Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.496Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.539Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.572Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.622Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.661Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.698Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.762Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.809Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.862Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.916Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.962Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:30.997Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:31.042Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:31.086Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:31.129Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:31.179Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:31.234Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:31.279Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:31.324Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:31.375Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:31.425Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:31.476Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:31.530Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:31.580Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:31.624Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:32.210Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:32.263Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:32.312Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:32.325Z: Starting 4 workers in us-central1-b...
Nov 14, 2018 6:56:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:32.680Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 14, 2018 6:56:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:56:46.206Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 14, 2018 6:57:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:57:19.293Z: Workers have started successfully.
Nov 14, 2018 6:57:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:57:33.003Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 14, 2018 6:57:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:57:33.046Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 14, 2018 6:57:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:57:38.938Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 14, 2018 6:57:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:57:50.717Z: Workers have started successfully.
Nov 14, 2018 6:58:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:58:42.333Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 14, 2018 6:58:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:58:42.460Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 14, 2018 6:58:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:58:43.238Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 14, 2018 6:58:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:58:43.370Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 14, 2018 6:58:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:58:44.961Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 14, 2018 6:58:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:58:45.072Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 14, 2018 6:58:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:58:48.573Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 14, 2018 6:58:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:58:48.830Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 14, 2018 6:59:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:59:17.770Z: Cleaning up.
Nov 14, 2018 6:59:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:59:17.891Z: Stopping worker pool...
Nov 14, 2018 7:00:21 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-14T19:00:21.666Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-14T19:00:21.715Z RUNNING Query7
2018-11-14T19:00:23.137Z new perf {"snapshots":null,"runtimeSec":96.769,"numEvents":10000000,"eventsPerSec":103338.87918651634,"eventBytesPerSec":1.3264215781913629E7,"numResults":100,"resultsPerSec":1.0333887918651634,"resultBytesPerSec":104.40326964213746,"startupDelaySec":-162.3,"processingDelaySec":71.215,"shutdownDelaySec":65.531,"timeDilation":10.162345379202016,"errors":null,"jobId":null}
2018-11-14T19:00:38.264Z RUNNING Query7
2018-11-14T19:00:40.084Z no activity
2018-11-14T19:00:55.256Z RUNNING Query7
2018-11-14T19:00:56.799Z no activity
2018-11-14T19:01:11.877Z RUNNING Query7
2018-11-14T19:01:13.903Z no activity
2018-11-14T19:01:28.971Z RUNNING Query7
2018-11-14T19:01:31.003Z no activity
2018-11-14T19:01:46.053Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 2,5,main]) completed. Took 5 mins 44.479 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 23s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/urnokppenxngq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1029
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1029/display/redirect?page=changes>
Changes:
[lcwik] Remove some unnecessary catch blocks (#7034)
------------------------------------------
[...truncated 263.45 KB...]
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 14, 2018 6:39:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-75nu4ZR7YTGUBfCO8vrYuA.jar
Nov 14, 2018 6:39:09 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-75nu4ZR7YTGUBfCO8vrYuA.jar
Nov 14, 2018 6:39:17 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-75nu4ZR7YTGUBfCO8vrYuA.jar
Nov 14, 2018 6:39:20 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 114 files cached, 29 files newly uploaded
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 14, 2018 6:39:21 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88181 bytes, hash FYFTKV6089NIYmPq7hB1sA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-FYFTKV6089NIYmPq7hB1sA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 14, 2018 6:39:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-14_10_39_22-7145344990643400530
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-14_10_39_22-7145344990643400530?project=apache-beam-testing
Nov 14, 2018 6:39:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-14_10_39_22-7145344990643400530
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-14T18:39:22.223Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:26.191Z: Checking permissions granted to controller Service Account.
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:30.696Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:31.377Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:31.536Z: Expanding GroupByKey operations into optimizable parts.
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:31.595Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:31.840Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:31.893Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#e0089b3c1ab3aafb
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:31.954Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:31.993Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.043Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.078Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.114Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.137Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.179Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.220Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.254Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.295Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.332Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.374Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.420Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.465Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.506Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.555Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.597Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.683Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.777Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.882Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:32.949Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:33.030Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:33.088Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:33.133Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:33.181Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:33.214Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:33.259Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:33.301Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:33.348Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:33.384Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:33.418Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 6:39:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:33.458Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 14, 2018 6:39:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:33.935Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 14, 2018 6:39:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:33.978Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 14, 2018 6:39:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:34.022Z: Starting 4 workers in us-central1-b...
Nov 14, 2018 6:39:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:34.037Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 14, 2018 6:39:36 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:34.349Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 14, 2018 6:39:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:39:51.695Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 14, 2018 6:40:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:40:19.216Z: Workers have started successfully.
Nov 14, 2018 6:40:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:40:31.498Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 14, 2018 6:40:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:40:31.543Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 14, 2018 6:40:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:40:37.613Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 14, 2018 6:40:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:40:49.782Z: Workers have started successfully.
Nov 14, 2018 6:41:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:41:47.466Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 14, 2018 6:41:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:41:47.607Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 14, 2018 6:41:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:41:49.381Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 14, 2018 6:41:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:41:49.460Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 14, 2018 6:41:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:41:50.850Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 14, 2018 6:41:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:41:50.969Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 14, 2018 6:41:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:41:54.660Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 14, 2018 6:41:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:41:54.903Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 14, 2018 6:42:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:42:22.325Z: Cleaning up.
Nov 14, 2018 6:42:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:42:22.481Z: Stopping worker pool...
Nov 14, 2018 6:43:23 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-14T18:43:23.151Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-14T18:43:23.266Z RUNNING Query7
2018-11-14T18:43:25.916Z new perf {"snapshots":null,"runtimeSec":105.637,"numEvents":10000000,"eventsPerSec":94663.80150894099,"eventBytesPerSec":1.2150713263345229E7,"numResults":100,"resultsPerSec":0.9466380150894099,"resultBytesPerSec":95.63883866448309,"startupDelaySec":-167.136,"processingDelaySec":80.757,"shutdownDelaySec":61.499,"timeDilation":9.309238240389258,"errors":null,"jobId":null}
2018-11-14T18:43:41.004Z RUNNING Query7
2018-11-14T18:43:42.930Z no activity
2018-11-14T18:43:58.059Z RUNNING Query7
2018-11-14T18:44:00.036Z no activity
2018-11-14T18:44:15.080Z RUNNING Query7
2018-11-14T18:44:17.139Z no activity
2018-11-14T18:44:32.289Z RUNNING Query7
2018-11-14T18:44:34.336Z no activity
2018-11-14T18:44:49.404Z RUNNING Query7
2018-11-14T18:44:51.018Z no activity
2018-11-14T18:45:06.068Z RUNNING Query7
2018-11-14T18:45:09.230Z no activity
2018-11-14T18:45:24.315Z RUNNING Query7
2018-11-14T18:45:26.159Z no activity
2018-11-14T18:45:41.262Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 2,5,main]) completed. Took 6 mins 47.703 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 33s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/kfi3lbf5mdi6s
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1028
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1028/display/redirect>
------------------------------------------
[...truncated 249.60 KB...]
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:10.836Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:10.872Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:10.907Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:10.936Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:10.974Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:11.007Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:11.038Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:11.072Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:11.122Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:11.159Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:11.206Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:11.243Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:11.279Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:11.322Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:11.361Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:11.396Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:11.429Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:11.465Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:11.499Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:11.540Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:11.570Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:12.008Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:12.045Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:12.076Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:12.090Z: Starting 4 workers in us-central1-b...
Nov 14, 2018 6:04:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:12.304Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 14, 2018 6:04:25 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:04:23.349Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 14, 2018 6:05:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:05:41.214Z: Workers have started successfully.
Nov 14, 2018 6:05:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:05:45.013Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 14, 2018 6:05:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:05:45.053Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 14, 2018 6:05:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:05:49.235Z: Workers have started successfully.
Nov 14, 2018 6:05:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:05:50.861Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 14, 2018 6:05:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:05:50.899Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 14, 2018 6:06:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:06:42.461Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 14, 2018 6:07:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:07:12.334Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 14, 2018 6:07:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:07:12.442Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 14, 2018 6:07:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:07:14.571Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 14, 2018 6:07:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:07:14.666Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 14, 2018 6:07:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:07:16.264Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 14, 2018 6:07:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:07:16.440Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 14, 2018 6:07:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:07:20.671Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 14, 2018 6:07:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:07:20.871Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 14, 2018 6:07:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:07:47.412Z: Cleaning up.
Nov 14, 2018 6:07:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T18:07:47.572Z: Stopping worker pool...
Nov 14, 2018 6:08:00 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-14T18:08:00.499Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-14T18:08:00.552Z RUNNING Query7
2018-11-14T18:08:01.966Z new perf {"snapshots":null,"runtimeSec":110.971,"numEvents":10000000,"eventsPerSec":90113.63329158069,"eventBytesPerSec":1.1566669643420354E7,"numResults":100,"resultsPerSec":0.9011363329158067,"resultBytesPerSec":91.04180371448396,"startupDelaySec":-124.778,"processingDelaySec":87.467,"shutdownDelaySec":13.807,"timeDilation":8.861774697894043,"errors":null,"jobId":null}
2018-11-14T18:08:17.064Z RUNNING Query7
2018-11-14T18:08:19.191Z no activity
2018-11-14T18:08:34.308Z RUNNING Query7
2018-11-14T18:08:36.175Z no activity
2018-11-14T18:08:51.293Z RUNNING Query7
2018-11-14T18:08:52.596Z no activity
2018-11-14T18:09:07.640Z RUNNING Query7
2018-11-14T18:09:09.441Z no activity
2018-11-14T18:09:24.502Z RUNNING Query7
2018-11-14T18:09:26.019Z no activity
2018-11-14T18:09:41.175Z RUNNING Query7
2018-11-14T18:09:42.880Z no activity
2018-11-14T18:09:57.921Z RUNNING Query7
2018-11-14T18:09:59.218Z no activity
2018-11-14T18:10:14.271Z RUNNING Query7
2018-11-14T18:10:15.687Z no activity
2018-11-14T18:10:30.871Z RUNNING Query7
2018-11-14T18:10:32.944Z no activity
2018-11-14T18:10:48.030Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Daemon worker,5,main]) completed. Took 7 mins 0.003 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 40s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/f6v6ijzmmopdy
Build cache (/home/jenkins/.gradle/caches/build-cache-1) removing files not accessed on or after Wed Nov 07 18:00:14 UTC 2018.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4358} MaxSize{4500}, CacheStats{hitCount=3, missCount=4004, loadSuccessCount=4005, loadExceptionCount=0, totalLoadTime=352202029, evictionCount=1}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=4596, loadSuccessCount=4597, loadExceptionCount=0, totalLoadTime=392024152, evictionCount=451}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=5046, loadSuccessCount=5047, loadExceptionCount=0, totalLoadTime=419337806, evictionCount=901}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=5496, loadSuccessCount=5497, loadExceptionCount=0, totalLoadTime=448745746, evictionCount=1351}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=5946, loadSuccessCount=5947, loadExceptionCount=0, totalLoadTime=480330653, evictionCount=1801}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=6396, loadSuccessCount=6397, loadExceptionCount=0, totalLoadTime=516100574, evictionCount=2251}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=6846, loadSuccessCount=6847, loadExceptionCount=0, totalLoadTime=543852381, evictionCount=2701}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=7296, loadSuccessCount=7297, loadExceptionCount=0, totalLoadTime=580018095, evictionCount=3151}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=7746, loadSuccessCount=7747, loadExceptionCount=0, totalLoadTime=611595482, evictionCount=3601}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=8196, loadSuccessCount=8197, loadExceptionCount=0, totalLoadTime=649023735, evictionCount=4051}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=8646, loadSuccessCount=8647, loadExceptionCount=0, totalLoadTime=687653300, evictionCount=4501}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=9096, loadSuccessCount=9097, loadExceptionCount=0, totalLoadTime=718851060, evictionCount=4951}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=9546, loadSuccessCount=9547, loadExceptionCount=0, totalLoadTime=744300299, evictionCount=5401}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=9996, loadSuccessCount=9997, loadExceptionCount=0, totalLoadTime=771485824, evictionCount=5851}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=10446, loadSuccessCount=10447, loadExceptionCount=0, totalLoadTime=798358086, evictionCount=6301}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=10896, loadSuccessCount=10897, loadExceptionCount=0, totalLoadTime=827080185, evictionCount=6751}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=11346, loadSuccessCount=11347, loadExceptionCount=0, totalLoadTime=851941210, evictionCount=7201}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=11796, loadSuccessCount=11797, loadExceptionCount=0, totalLoadTime=873917040, evictionCount=7651}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=12246, loadSuccessCount=12247, loadExceptionCount=0, totalLoadTime=901003641, evictionCount=8101}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=12696, loadSuccessCount=12697, loadExceptionCount=0, totalLoadTime=922449946, evictionCount=8551}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=13146, loadSuccessCount=13147, loadExceptionCount=0, totalLoadTime=946674048, evictionCount=9001}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=13596, loadSuccessCount=13597, loadExceptionCount=0, totalLoadTime=970472531, evictionCount=9451}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=14046, loadSuccessCount=14047, loadExceptionCount=0, totalLoadTime=998756640, evictionCount=9901}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=14496, loadSuccessCount=14497, loadExceptionCount=0, totalLoadTime=1027238206, evictionCount=10351}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=14946, loadSuccessCount=14947, loadExceptionCount=0, totalLoadTime=1054698722, evictionCount=10801}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=15396, loadSuccessCount=15397, loadExceptionCount=0, totalLoadTime=1077937894, evictionCount=11251}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=15846, loadSuccessCount=15847, loadExceptionCount=0, totalLoadTime=1104638388, evictionCount=11701}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=16296, loadSuccessCount=16297, loadExceptionCount=0, totalLoadTime=1129355876, evictionCount=12151}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=16746, loadSuccessCount=16747, loadExceptionCount=0, totalLoadTime=1153475424, evictionCount=12601}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=17196, loadSuccessCount=17197, loadExceptionCount=0, totalLoadTime=1181099175, evictionCount=13051}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=17646, loadSuccessCount=17647, loadExceptionCount=0, totalLoadTime=1207290069, evictionCount=13501}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=18096, loadSuccessCount=18097, loadExceptionCount=0, totalLoadTime=1233172678, evictionCount=13951}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=18546, loadSuccessCount=18547, loadExceptionCount=0, totalLoadTime=1265855875, evictionCount=14401}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=18996, loadSuccessCount=18997, loadExceptionCount=0, totalLoadTime=1291450337, evictionCount=14851}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=19446, loadSuccessCount=19447, loadExceptionCount=0, totalLoadTime=1312668728, evictionCount=15301}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=19896, loadSuccessCount=19897, loadExceptionCount=0, totalLoadTime=1336587377, evictionCount=15751}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=20346, loadSuccessCount=20347, loadExceptionCount=0, totalLoadTime=1363738973, evictionCount=16201}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=20796, loadSuccessCount=20797, loadExceptionCount=0, totalLoadTime=1391607703, evictionCount=16651}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=21246, loadSuccessCount=21247, loadExceptionCount=0, totalLoadTime=1420681695, evictionCount=17101}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=21696, loadSuccessCount=21697, loadExceptionCount=0, totalLoadTime=1450459880, evictionCount=17551}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=22146, loadSuccessCount=22147, loadExceptionCount=0, totalLoadTime=1482763365, evictionCount=18001}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=22596, loadSuccessCount=22597, loadExceptionCount=0, totalLoadTime=1508006129, evictionCount=18451}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=23046, loadSuccessCount=23047, loadExceptionCount=0, totalLoadTime=1531664635, evictionCount=18901}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=23496, loadSuccessCount=23497, loadExceptionCount=0, totalLoadTime=1558334932, evictionCount=19351}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=23946, loadSuccessCount=23947, loadExceptionCount=0, totalLoadTime=1587293763, evictionCount=19801}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=24396, loadSuccessCount=24397, loadExceptionCount=0, totalLoadTime=1608290063, evictionCount=20251}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=24846, loadSuccessCount=24847, loadExceptionCount=0, totalLoadTime=1637015712, evictionCount=20701}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=25296, loadSuccessCount=25297, loadExceptionCount=0, totalLoadTime=1663202722, evictionCount=21151}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=25746, loadSuccessCount=25747, loadExceptionCount=0, totalLoadTime=1691306257, evictionCount=21601}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=26196, loadSuccessCount=26197, loadExceptionCount=0, totalLoadTime=1717581446, evictionCount=22051}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=26646, loadSuccessCount=26647, loadExceptionCount=0, totalLoadTime=1743191887, evictionCount=22501}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=27096, loadSuccessCount=27097, loadExceptionCount=0, totalLoadTime=1774057524, evictionCount=22951}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=27546, loadSuccessCount=27547, loadExceptionCount=0, totalLoadTime=1802737994, evictionCount=23401}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Cache entries evicted. In-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin: Size{4500} MaxSize{4500}, CacheStats{hitCount=3, missCount=27996, loadSuccessCount=27997, loadExceptionCount=0, totalLoadTime=1830089634, evictionCount=23851}
Performance may suffer from in-memory cache misses. Increase max heap size of Gradle build process to reduce cache misses.
Build cache (/home/jenkins/.gradle/caches/build-cache-1) cleaned up in 2.568 secs.
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1027
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1027/display/redirect?page=changes>
Changes:
[echauchot] [BEAM-5813] wait for the refresh of ES indices in all test utils that do
------------------------------------------
[...truncated 249.92 KB...]
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 14, 2018 4:14:27 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-IaplrIV0PpCBLWCeIfGy1g.jar
Nov 14, 2018 4:14:30 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-IaplrIV0PpCBLWCeIfGy1g.jar
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 114 files cached, 29 files newly uploaded
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 14, 2018 4:14:34 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 14, 2018 4:14:35 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88187 bytes, hash Vf_6nGB6SBSGtg-QYK41zA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-Vf_6nGB6SBSGtg-QYK41zA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 14, 2018 4:14:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-14_08_14_35-7078439089365617470
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-14_08_14_35-7078439089365617470?project=apache-beam-testing
Nov 14, 2018 4:14:36 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-14_08_14_35-7078439089365617470
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-14T16:14:35.450Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:38.464Z: Checking permissions granted to controller Service Account.
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:43.361Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:43.851Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:43.954Z: Expanding GroupByKey operations into optimizable parts.
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:43.996Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:44.206Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:44.258Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#37bc5e2e3faec20a
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:44.299Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:44.346Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:44.398Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:44.447Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:44.499Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:44.549Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:44.600Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:44.651Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:44.706Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:44.761Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:44.807Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:44.852Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:44.903Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:44.953Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.003Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.044Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.099Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.152Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.191Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.240Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.290Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.338Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.391Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.439Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.487Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.541Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.591Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.633Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.674Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.724Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.764Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:45.817Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:46.294Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:46.334Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:46.377Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:46.395Z: Starting 4 workers in us-central1-b...
Nov 14, 2018 4:14:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:47.481Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 14, 2018 4:15:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:14:59.898Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 14, 2018 4:15:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:15:49.210Z: Workers have started successfully.
Nov 14, 2018 4:15:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:15:51.900Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s).
Nov 14, 2018 4:15:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:15:52.019Z: Resized worker pool to 1, though goal was 4. This could be a quota issue.
Nov 14, 2018 4:15:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:15:58.928Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 14, 2018 4:15:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:15:58.972Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 14, 2018 4:16:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:16:04.976Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 14, 2018 4:16:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:16:19.562Z: Workers have started successfully.
Nov 14, 2018 4:17:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:17:09.816Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 14, 2018 4:17:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:17:09.898Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 14, 2018 4:17:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:17:11.278Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 14, 2018 4:17:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:17:11.364Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 14, 2018 4:17:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:17:11.836Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 14, 2018 4:17:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:17:11.923Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 14, 2018 4:17:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:17:15.872Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 14, 2018 4:17:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:17:16.078Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 14, 2018 4:17:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:17:44.332Z: Cleaning up.
Nov 14, 2018 4:17:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T16:17:44.546Z: Stopping worker pool...
Nov 14, 2018 4:18:36 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-14T16:18:36.570Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-14T16:18:36.647Z RUNNING Query7
2018-11-14T16:18:38.110Z new perf {"snapshots":null,"runtimeSec":99.246,"numEvents":10000000,"eventsPerSec":100759.72835177237,"eventBytesPerSec":1.2933165034359068E7,"numResults":100,"resultsPerSec":1.0075972835177236,"resultBytesPerSec":101.79755355379562,"startupDelaySec":-151.867,"processingDelaySec":73.352,"shutdownDelaySec":52.622,"timeDilation":9.908711686113294,"errors":null,"jobId":null}
2018-11-14T16:18:53.252Z RUNNING Query7
2018-11-14T16:18:55.052Z no activity
2018-11-14T16:19:10.254Z RUNNING Query7
2018-11-14T16:19:11.554Z no activity
2018-11-14T16:19:26.600Z RUNNING Query7
2018-11-14T16:19:28.326Z no activity
2018-11-14T16:19:43.394Z RUNNING Query7
2018-11-14T16:19:44.706Z no activity
2018-11-14T16:19:59.766Z RUNNING Query7
2018-11-14T16:20:00.895Z no activity
2018-11-14T16:20:15.944Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 7,5,main]) completed. Took 6 mins 4.378 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 46s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/aapdchzpnmilu
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1026
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1026/display/redirect>
------------------------------------------
[...truncated 257.46 KB...]
} ],
"message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-PM_r0wNgIUYjnsw3STy5Nw.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 14, 2018 12:04:15 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-PM_r0wNgIUYjnsw3STy5Nw.jar
Nov 14, 2018 12:04:18 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 114 files cached, 28 files newly uploaded
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 14, 2018 12:04:19 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88179 bytes, hash RdAgJrOcVe4iHBt-_K9JVg> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-RdAgJrOcVe4iHBt-_K9JVg.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 14, 2018 12:04:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-14_04_04_19-11230185407420314233?project=apache-beam-testing
Submitted job: 2018-11-14_04_04_19-11230185407420314233
Nov 14, 2018 12:04:20 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-14_04_04_19-11230185407420314233
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-14T12:04:19.864Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:22.612Z: Checking permissions granted to controller Service Account.
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:28.766Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:29.247Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:29.320Z: Expanding GroupByKey operations into optimizable parts.
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:29.360Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:29.577Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:29.629Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:29.647Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:29.691Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:29.788Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:29.836Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:29.892Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:29.945Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.005Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.045Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.072Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.122Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.166Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.215Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.262Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.314Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.352Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.389Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.440Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.489Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.518Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.566Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.603Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.652Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.699Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.740Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.792Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.839Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.878Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.911Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:30.965Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:31.010Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:31.055Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:31.104Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:31.579Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:31.628Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:31.678Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 14, 2018 12:04:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:31.678Z: Starting 4 workers in us-central1-b...
Nov 14, 2018 12:04:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:31.940Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 14, 2018 12:04:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:04:48.576Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 14, 2018 12:05:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:05:11.439Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 14, 2018 12:05:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:05:11.474Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 14, 2018 12:05:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:05:16.549Z: Workers have started successfully.
Nov 14, 2018 12:05:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:05:16.581Z: Workers have started successfully.
Nov 14, 2018 12:05:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:05:28.075Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 14, 2018 12:06:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:06:41.637Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 14, 2018 12:06:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:06:41.710Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 14, 2018 12:06:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:06:43.591Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 14, 2018 12:06:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:06:43.672Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 14, 2018 12:06:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:06:44.913Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 14, 2018 12:06:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:06:44.996Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 14, 2018 12:06:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:06:47.951Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 14, 2018 12:06:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:06:48.147Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 14, 2018 12:07:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:07:14.893Z: Cleaning up.
Nov 14, 2018 12:07:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T12:07:14.994Z: Stopping worker pool...
Nov 14, 2018 12:08:20 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-14T12:08:20.721Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-14T12:08:20.766Z RUNNING Query7
2018-11-14T12:08:21.889Z new perf {"snapshots":null,"runtimeSec":103.276,"numEvents":10000000,"eventsPerSec":96827.91742515202,"eventBytesPerSec":1.2428491585653976E7,"numResults":100,"resultsPerSec":0.9682791742515202,"resultBytesPerSec":97.82524497463109,"startupDelaySec":-169.667,"processingDelaySec":79.147,"shutdownDelaySec":66.391,"timeDilation":9.52205739958945,"errors":null,"jobId":null}
2018-11-14T12:08:36.954Z RUNNING Query7
2018-11-14T12:08:38.098Z no activity
2018-11-14T12:08:53.201Z RUNNING Query7
2018-11-14T12:08:54.392Z no activity
2018-11-14T12:09:09.455Z RUNNING Query7
2018-11-14T12:09:10.586Z no activity
2018-11-14T12:09:25.652Z RUNNING Query7
2018-11-14T12:09:26.687Z no activity
2018-11-14T12:09:41.732Z RUNNING Query7
2018-11-14T12:09:42.815Z no activity
2018-11-14T12:09:57.860Z RUNNING Query7
2018-11-14T12:09:58.517Z no activity
2018-11-14T12:10:13.517Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 3,5,main]) completed. Took 6 mins 24.375 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 6s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/lf5epa7l3vy3c
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1025
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1025/display/redirect?page=changes>
Changes:
[robertwb] Use sha256 instead of insecure md5 for artifact checksums.
------------------------------------------
[...truncated 231.92 KB...]
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-bCDFD3MIz1GzLvlLt41iaA.jar
Nov 14, 2018 9:41:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-bCDFD3MIz1GzLvlLt41iaA.jar
Nov 14, 2018 9:41:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-bCDFD3MIz1GzLvlLt41iaA.jar
Nov 14, 2018 9:41:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-bCDFD3MIz1GzLvlLt41iaA.jar
Nov 14, 2018 9:41:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-bCDFD3MIz1GzLvlLt41iaA.jar
Nov 14, 2018 9:41:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-bCDFD3MIz1GzLvlLt41iaA.jar
Nov 14, 2018 9:41:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-bCDFD3MIz1GzLvlLt41iaA.jar
Nov 14, 2018 9:41:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-bCDFD3MIz1GzLvlLt41iaA.jar
Nov 14, 2018 9:41:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-bCDFD3MIz1GzLvlLt41iaA.jar
Nov 14, 2018 9:41:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-bCDFD3MIz1GzLvlLt41iaA.jar
Nov 14, 2018 9:41:47 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/vendor/grpc-v1_13_1/build/libs/beam-vendor-grpc-v1_13_1-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-vendor-grpc-v1_13_1-2.9.0-SNAPSHOT-tT7ZtzYqxubhKeBmTYA6yQ.jar
Nov 14, 2018 9:41:50 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 114 files cached, 29 files newly uploaded
Nov 14, 2018 9:41:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 14, 2018 9:41:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 14, 2018 9:41:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 14, 2018 9:41:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 14, 2018 9:41:50 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 14, 2018 9:41:51 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88162 bytes, hash SdH3R7b-TdqX5rB5TPgkwA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-SdH3R7b-TdqX5rB5TPgkwA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-14_01_41_51-12143332912238019483
Nov 14, 2018 9:41:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-14_01_41_51-12143332912238019483?project=apache-beam-testing
Nov 14, 2018 9:41:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-14_01_41_51-12143332912238019483
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-14T09:41:51.728Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:41:55.257Z: Checking permissions granted to controller Service Account.
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:03.540Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:04.037Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:04.137Z: Expanding GroupByKey operations into optimizable parts.
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:04.181Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:04.432Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:04.483Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#8bc528a076ecd5d7
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:04.539Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:04.580Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:04.664Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:04.712Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:04.763Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:04.807Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:04.852Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:04.897Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:04.937Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:04.971Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.010Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.061Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.105Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.139Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.178Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.220Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.273Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.388Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.477Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.549Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.602Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.646Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.684Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.721Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.762Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.819Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.849Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.885Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.921Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:05.960Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:06.000Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:06.035Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:06.513Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:06.554Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:06.598Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:06.612Z: Starting 4 workers in us-central1-b...
Nov 14, 2018 9:42:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:07.130Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 14, 2018 9:42:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:42:28.308Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 14, 2018 9:43:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:43:06.493Z: Workers have started successfully.
Nov 14, 2018 9:43:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:43:14.794Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 14, 2018 9:43:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:43:37.697Z: Workers have started successfully.
Nov 14, 2018 9:44:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:44:25.139Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 14, 2018 9:44:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:44:25.260Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 14, 2018 9:44:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:44:27.667Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 14, 2018 9:44:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:44:27.795Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 14, 2018 9:44:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:44:29.785Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 14, 2018 9:44:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:44:29.897Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 14, 2018 9:44:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:44:34.381Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 14, 2018 9:44:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:44:34.621Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 14, 2018 9:45:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:45:01.888Z: Cleaning up.
Nov 14, 2018 9:45:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T09:45:01.996Z: Stopping worker pool...
Nov 14, 2018 9:45:52 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-14T09:45:52.610Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-14T09:45:52.671Z RUNNING Query7
2018-11-14T09:45:54.021Z new perf {"snapshots":null,"runtimeSec":101.446,"numEvents":10000000,"eventsPerSec":98574.61112315912,"eventBytesPerSec":1.265269105731128E7,"numResults":100,"resultsPerSec":0.9857461112315912,"resultBytesPerSec":99.58992961772766,"startupDelaySec":-152.645,"processingDelaySec":76.359,"shutdownDelaySec":51.199,"timeDilation":9.693827257851467,"errors":null,"jobId":null}
2018-11-14T09:46:09.075Z RUNNING Query7
2018-11-14T09:46:10.242Z no activity
2018-11-14T09:46:25.284Z RUNNING Query7
2018-11-14T09:46:26.329Z no activity
2018-11-14T09:46:41.364Z RUNNING Query7
2018-11-14T09:46:42.445Z no activity
2018-11-14T09:46:57.484Z RUNNING Query7
2018-11-14T09:46:58.498Z no activity
2018-11-14T09:47:13.552Z RUNNING Query7
2018-11-14T09:47:14.592Z no activity
2018-11-14T09:47:29.652Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 11,5,main]) completed. Took 5 mins 47.123 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 15s
76 actionable tasks: 72 executed, 4 from cache
Publishing build scan...
https://gradle.com/s/ejuzjsj2lp2va
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1024
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1024/display/redirect>
------------------------------------------
[...truncated 231.30 KB...]
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/vendor/grpc-v1_13_1/build/libs/beam-vendor-grpc-v1_13_1-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-vendor-grpc-v1_13_1-2.9.0-SNAPSHOT-jEurOmB28UkTtcVPcPtalA.jar
Nov 14, 2018 6:03:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-_P3emFQNtx56esuSX6v1cw.jar
Nov 14, 2018 6:03:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-_P3emFQNtx56esuSX6v1cw.jar
Nov 14, 2018 6:03:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-_P3emFQNtx56esuSX6v1cw.jar
Nov 14, 2018 6:03:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-_P3emFQNtx56esuSX6v1cw.jar
Nov 14, 2018 6:03:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-_P3emFQNtx56esuSX6v1cw.jar
Nov 14, 2018 6:03:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-_P3emFQNtx56esuSX6v1cw.jar
Nov 14, 2018 6:03:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-_P3emFQNtx56esuSX6v1cw.jar
Nov 14, 2018 6:03:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-_P3emFQNtx56esuSX6v1cw.jar
Nov 14, 2018 6:03:42 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-_P3emFQNtx56esuSX6v1cw.jar
Nov 14, 2018 6:03:45 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 114 files cached, 29 files newly uploaded
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 14, 2018 6:03:46 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88172 bytes, hash DchozabaqXIPXl7FxxohpA> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-DchozabaqXIPXl7FxxohpA.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Submitted job: 2018-11-13_22_03_46-11329820390924945951
Nov 14, 2018 6:03:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-13_22_03_46-11329820390924945951?project=apache-beam-testing
Nov 14, 2018 6:03:47 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-13_22_03_46-11329820390924945951
Nov 14, 2018 6:04:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-14T06:03:47.119Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 14, 2018 6:04:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:49.872Z: Checking permissions granted to controller Service Account.
Nov 14, 2018 6:04:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:57.970Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 14, 2018 6:04:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:58.503Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 14, 2018 6:04:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:58.598Z: Expanding GroupByKey operations into optimizable parts.
Nov 14, 2018 6:04:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:58.648Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 14, 2018 6:04:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:58.851Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 14, 2018 6:04:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:58.894Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#8bc528a076ecd5d7
Nov 14, 2018 6:04:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:58.940Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:58.986Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.017Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.055Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.093Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.132Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.180Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.222Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.255Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.299Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.339Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.375Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.424Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.473Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.514Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.549Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.593Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.643Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.685Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.722Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.761Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.800Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.843Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.880Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.920Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:03:59.978Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:04:00.022Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:04:00.060Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:04:00.115Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:04:00.153Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:04:00.194Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:04:00.232Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:04:00.720Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:04:00.770Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:04:00.814Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:04:00.827Z: Starting 4 workers in us-central1-b...
Nov 14, 2018 6:04:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:04:01.089Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 14, 2018 6:04:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:04:14.760Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 14, 2018 6:04:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:04:37.941Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s).
Nov 14, 2018 6:04:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:04:37.985Z: Resized worker pool to 2, though goal was 4. This could be a quota issue.
Nov 14, 2018 6:04:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:04:41.270Z: Workers have started successfully.
Nov 14, 2018 6:04:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:04:41.325Z: Workers have started successfully.
Nov 14, 2018 6:04:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:04:56.002Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 14, 2018 6:06:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:06:04.760Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 14, 2018 6:06:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:06:04.875Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 14, 2018 6:06:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:06:06.573Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 14, 2018 6:06:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:06:06.763Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 14, 2018 6:06:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:06:08.309Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 14, 2018 6:06:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:06:08.416Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 14, 2018 6:06:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:06:13.323Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 14, 2018 6:06:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:06:13.544Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 14, 2018 6:06:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:06:42.095Z: Cleaning up.
Nov 14, 2018 6:06:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T06:06:42.200Z: Stopping worker pool...
Nov 14, 2018 6:07:48 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-14T06:07:48.038Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-14T06:07:48.111Z RUNNING Query7
2018-11-14T06:07:49.341Z new perf {"snapshots":null,"runtimeSec":106.489,"numEvents":10000000,"eventsPerSec":93906.41286893481,"eventBytesPerSec":1.205349751617538E7,"numResults":100,"resultsPerSec":0.9390641286893482,"resultBytesPerSec":94.87364892148484,"startupDelaySec":-172.959,"processingDelaySec":80.034,"shutdownDelaySec":66.47,"timeDilation":9.23475664153105,"errors":null,"jobId":null}
2018-11-14T06:08:04.399Z RUNNING Query7
2018-11-14T06:08:05.577Z no activity
2018-11-14T06:08:20.618Z RUNNING Query7
2018-11-14T06:08:21.696Z no activity
2018-11-14T06:08:36.762Z RUNNING Query7
2018-11-14T06:08:37.928Z no activity
2018-11-14T06:08:52.966Z RUNNING Query7
2018-11-14T06:08:54.386Z no activity
2018-11-14T06:09:09.452Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 11,5,main]) completed. Took 5 mins 32.19 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 9m 0s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/4p43dwjk6p47q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Java_Nexmark_Dataflow
#1023
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1023/display/redirect?page=changes>
Changes:
[ehudm] [BEAM-6047] Try to fix HDFS DNS issues
------------------------------------------
[...truncated 276.17 KB...]
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:358)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Nov 14, 2018 12:54:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Jm2b6FeFoot2G70S7ejoPQ.jar
Nov 14, 2018 12:54:37 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Jm2b6FeFoot2G70S7ejoPQ.jar
Nov 14, 2018 12:54:38 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-Jm2b6FeFoot2G70S7ejoPQ.jar
Nov 14, 2018 12:54:42 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements
INFO: Staging files complete: 114 files cached, 29 files newly uploaded
Nov 14, 2018 12:54:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.ReadBounded as step s1
Nov 14, 2018 12:54:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Monitor as step s2
Nov 14, 2018 12:54:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Snoop as step s3
Nov 14, 2018 12:54:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4
Nov 14, 2018 12:54:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/justBids/AsBid as step s5
Nov 14, 2018 12:54:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6
Nov 14, 2018 12:54:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/BidToPrice as step s7
Nov 14, 2018 12:54:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8
Nov 14, 2018 12:54:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9
Nov 14, 2018 12:54:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10
Nov 14, 2018 12:54:42 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7/Query7.Select as step s23
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Debug as step s24
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7/Query7.Stamp as step s25
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.Format as step s26
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Query7.DevNull as step s27
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/
Nov 14, 2018 12:54:43 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <88160 bytes, hash sr5g3zFBFpUjTD7n8NT3Yw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-sr5g3zFBFpUjTD7n8NT3Yw.pb
Dataflow SDK version: 2.9.0-SNAPSHOT
Nov 14, 2018 12:54:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
Submitted job: 2018-11-13_16_54_43-13057491920058226617
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-13_16_54_43-13057491920058226617?project=apache-beam-testing
Nov 14, 2018 12:54:44 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-13_16_54_43-13057491920058226617
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2018-11-14T00:54:43.696Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:47.574Z: Checking permissions granted to controller Service Account.
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:53.480Z: Worker configuration: n1-standard-1 in us-central1-b.
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:53.981Z: Expanding CoGroupByKey operations into optimizable parts.
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:54.111Z: Expanding GroupByKey operations into optimizable parts.
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:54.167Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:54.550Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:54.644Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:54.706Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:54.771Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:54.809Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:54.847Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:54.897Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:54.936Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:54.985Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.040Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.078Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.130Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.177Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.215Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.266Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.311Z: Fusing consumer Query7.Format into Query7/Query7.Stamp
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.351Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.381Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.431Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.471Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.511Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.579Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.626Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.690Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous)
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.738Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.780Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.826Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.871Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.927Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:55.966Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:56.007Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:56.086Z: Fusing consumer Query7.DevNull into Query7.Format
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:56.141Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:56.182Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:56.857Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:56.895Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:56.921Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:56.944Z: Starting 4 workers in us-central1-b...
Nov 14, 2018 12:55:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:54:57.382Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write
Nov 14, 2018 12:55:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:55:15.623Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s).
Nov 14, 2018 12:55:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:55:54.918Z: Workers have started successfully.
Nov 14, 2018 12:55:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:55:56.976Z: Autoscaling: Raised the number of workers to 3 based on the rate of progress in the currently running step(s).
Nov 14, 2018 12:55:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:55:57.033Z: Resized worker pool to 3, though goal was 4. This could be a quota issue.
Nov 14, 2018 12:56:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:56:02.981Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s).
Nov 14, 2018 12:56:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:56:25.210Z: Workers have started successfully.
Nov 14, 2018 12:57:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:57:11.922Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close
Nov 14, 2018 12:57:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:57:12.082Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write
Nov 14, 2018 12:57:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:57:14.120Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close
Nov 14, 2018 12:57:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:57:14.306Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Nov 14, 2018 12:57:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:57:15.893Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Nov 14, 2018 12:57:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:57:16.096Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Nov 14, 2018 12:57:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:57:19.827Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView
Nov 14, 2018 12:57:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:57:20.094Z: Executing operation Query7/Query7/Query7.Select+Query7/Query7.Debug+Query7/Query7.Stamp+Query7.Format+Query7.DevNull
Nov 14, 2018 12:57:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:57:47.367Z: Cleaning up.
Nov 14, 2018 12:57:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-11-14T00:57:47.452Z: Stopping worker pool...
Nov 14, 2018 12:58:44 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
2018-11-14T00:58:44.734Z Waiting for main pipeline to 'finish'
WARNING: No terminal state was returned. State value RUNNING
2018-11-14T00:58:44.791Z RUNNING Query7
2018-11-14T00:58:46.018Z new perf {"snapshots":null,"runtimeSec":97.105,"numEvents":10000000,"eventsPerSec":102981.30889243602,"eventBytesPerSec":1.3218319314144483E7,"numResults":100,"resultsPerSec":1.0298130889243602,"resultBytesPerSec":104.04201637402811,"startupDelaySec":-155.06,"processingDelaySec":72.734,"shutdownDelaySec":57.956,"timeDilation":10.127181916482158,"errors":null,"jobId":null}
2018-11-14T00:59:01.092Z RUNNING Query7
2018-11-14T00:59:02.186Z no activity
2018-11-14T00:59:17.228Z RUNNING Query7
2018-11-14T00:59:19.210Z no activity
2018-11-14T00:59:34.258Z RUNNING Query7
2018-11-14T00:59:35.479Z no activity
2018-11-14T00:59:50.539Z RUNNING Query7
2018-11-14T00:59:51.616Z no activity
2018-11-14T01:00:06.656Z RUNNING Query7
2018-11-14T01:00:08.230Z no activity
2018-11-14T01:00:23.315Z RUNNING Query7
2018-11-14T01:00:24.602Z no activity
2018-11-14T01:00:39.663Z DONE Query7
> Task :beam-sdks-java-nexmark:run FAILED
:beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 4,5,main]) completed. Took 6 mins 28.77 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':beam-sdks-java-nexmark:run'.
> Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 10m 32s
76 actionable tasks: 71 executed, 5 from cache
Publishing build scan...
https://gradle.com/s/jrebds7jrkfsg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org