You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/05/18 06:39:00 UTC
Build failed in Jenkins: beam_PerformanceTests_ParquetIOIT #4376
See <https://ci-beam.apache.org/job/beam_PerformanceTests_ParquetIOIT/4376/display/redirect>
Changes:
------------------------------------------
[...truncated 319.32 KB...]
INFO: 2021-05-18T06:29:56.196Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate hashcode/WithKeys/AddKeys/Map
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.232Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.263Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.297Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.338Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.378Z: Fusing consumer Calculate hashcode/Values/Values/Map into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.411Z: Unzipping flatten s9-u129 for input s10.org.apache.beam.sdk.values.PCollection.<init>:402#2587af97b4865538-c127
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.447Z: Fusing unzipped copy of Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign, through flatten Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections/Unzipped-1, into producer Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.483Z: Fusing consumer Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.519Z: Fusing consumer Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign into Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.553Z: Fusing consumer Produce text lines into Generate sequence/Read(BoundedCountingSource)
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.587Z: Fusing consumer Produce Avro records into Produce text lines
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.620Z: Fusing consumer Gather write start times into Produce Avro records
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.654Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into Gather write start times
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.701Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.734Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.796Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.834Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.868Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.901Z: Fusing consumer PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) into Calculate hashcode/ProduceDefault
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.936Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map into Calculate hashcode/ProduceDefault
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:56.961Z: Fusing consumer Calculate hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:57.001Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/Reify into PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:57.033Z: Fusing consumer PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
May 18, 2021 6:29:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:57.609Z: Executing operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
May 18, 2021 6:30:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:57.706Z: Starting 5 ****s in us-central1-c...
May 18, 2021 6:30:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:58.230Z: Finished operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
May 18, 2021 6:30:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:58.400Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Create
May 18, 2021 6:30:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:58.612Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Create
May 18, 2021 6:30:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:58.638Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 18, 2021 6:30:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:29:58.797Z: Executing operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Produce Avro records+Gather write start times+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
May 18, 2021 6:30:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:30:48.283Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
May 18, 2021 6:31:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:31:16.638Z: Workers have started successfully.
May 18, 2021 6:31:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:31:16.667Z: Workers have started successfully.
May 18, 2021 6:34:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:23.858Z: Finished operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Produce Avro records+Gather write start times+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
May 18, 2021 6:34:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:23.936Z: Executing operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
May 18, 2021 6:34:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:24.095Z: Finished operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
May 18, 2021 6:34:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:24.169Z: Executing operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write
May 18, 2021 6:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:24.673Z: Finished operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write
May 18, 2021 6:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:24.728Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Close
May 18, 2021 6:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:24.810Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Close
May 18, 2021 6:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:24.897Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:25.110Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:25.294Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/GatherTempFileResults/Drop key/Values/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Gather bundles+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair with random key+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:26.704Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/GatherTempFileResults/Drop key/Values/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Gather bundles+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair with random key+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:26.788Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:26.896Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:26.960Z: Executing operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:27.188Z: Finished operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:27.397Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Values/Values/Map+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Finalize+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:34:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:29.322Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Values/Values/Map+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Finalize+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:34:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:29.403Z: Executing operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:34:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:29.472Z: Finished operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:34:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:29.560Z: Executing operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:34:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:29.803Z: Finished operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:34:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:29.969Z: Executing operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Gather write end times+Get file names/Values/Map+Find files/Match filepatterns+Find files/Reshuffle.ViaRandomKey/Pair with random key+Find files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:34:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:32.334Z: Finished operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Gather write end times+Get file names/Values/Map+Find files/Match filepatterns+Find files/Reshuffle.ViaRandomKey/Pair with random key+Find files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:34:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:32.441Z: Executing operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:34:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:32.505Z: Finished operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:34:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:32.578Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
May 18, 2021 6:34:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:32.836Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
May 18, 2021 6:34:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:34:33.022Z: Executing operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Find files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Find files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matched files/ParDo(ToReadableFile)+Gather read start time+Read parquet files/ParDo(Read)+Gather read end time+Map records to strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
May 18, 2021 6:36:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:34.717Z: Finished operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Find files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Find files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matched files/ParDo(ToReadableFile)+Gather read start time+Read parquet files/ParDo(Read)+Gather read end time+Map records to strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
May 18, 2021 6:36:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:34.814Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Close
May 18, 2021 6:36:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:35.381Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Close
May 18, 2021 6:36:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:35.465Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
May 18, 2021 6:36:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:35.501Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Create
May 18, 2021 6:36:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:35.693Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
May 18, 2021 6:36:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:35.693Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Create
May 18, 2021 6:36:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:35.856Z: Executing operation PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 18, 2021 6:36:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:35.904Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate hashcode/Values/Values/Map+Calculate hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 18, 2021 6:36:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:36.203Z: Finished operation PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 18, 2021 6:36:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:37.523Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate hashcode/Values/Values/Map+Calculate hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 18, 2021 6:36:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:37.678Z: Executing operation Calculate hashcode/View.AsIterable/CreateDataflowView
May 18, 2021 6:36:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:37.752Z: Finished operation Calculate hashcode/View.AsIterable/CreateDataflowView
May 18, 2021 6:36:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:37.929Z: Executing operation Calculate hashcode/CreateVoid/Read(CreateSource)+Calculate hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
May 18, 2021 6:36:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:38.540Z: Finished operation Calculate hashcode/CreateVoid/Read(CreateSource)+Calculate hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
May 18, 2021 6:36:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:38.613Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
May 18, 2021 6:36:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:38.650Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Close
May 18, 2021 6:36:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:38.669Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
May 18, 2021 6:36:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:38.708Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Close
May 18, 2021 6:36:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:38.735Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
May 18, 2021 6:36:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:38.779Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
May 18, 2021 6:36:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:39.012Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
May 18, 2021 6:36:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:39.168Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
May 18, 2021 6:36:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:39.672Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
May 18, 2021 6:36:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:39.735Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
May 18, 2021 6:36:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:39.801Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
May 18, 2021 6:36:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:39.885Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
May 18, 2021 6:36:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:39.957Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
May 18, 2021 6:36:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:41.703Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
May 18, 2021 6:36:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:41.880Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
May 18, 2021 6:36:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:41.930Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
May 18, 2021 6:36:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:42.065Z: Executing operation Delete test files
May 18, 2021 6:36:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:44.014Z: Finished operation Delete test files
May 18, 2021 6:36:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:44.209Z: Cleaning up.
May 18, 2021 6:36:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:36:44.323Z: Stopping **** pool...
May 18, 2021 6:37:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:37:37.921Z: Autoscaling: Resized **** pool from 5 to 0.
May 18, 2021 6:37:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T06:37:37.967Z: Worker pool stopped.
May 18, 2021 6:37:45 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-05-17_23_29_26-5964798609611105909 finished with status DONE.
org.apache.beam.sdk.io.parquet.ParquetIOIT > writeThenReadAll STANDARD_OUT
Load test results for test (ID): d5f51153-4e87-443a-a8bf-1796fb243de1 and timestamp: 2021-05-18T06:37:46.022000000Z:
Metric: Value:
read_time 121.195
dataset_size 1.08737E9
write_time 176.896
run_time 299.647
org.apache.beam.sdk.io.parquet.ParquetIOIT > writeThenReadAll STANDARD_ERROR
log4j:WARN No appenders could be found for logger (org.apache.http.client.protocol.RequestAddCookies).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest
Finished generating test XML results (0.046 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_ParquetIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.124 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_ParquetIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
Stored cache entry for task ':sdks:java:io:file-based-io-tests:integrationTest' with cache key c6cdf9f1786007343d9f2cd2271eeee2
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 9 mins 27.413 secs.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 13m 47s
120 actionable tasks: 73 executed, 47 from cache
Publishing build scan...
https://gradle.com/s/b67r5rdv2e5je
FAILURE: Build failed with an exception.
* What went wrong:
Timeout waiting to lock daemon addresses registry. It is currently in use by another Gradle instance.
Owner PID: unknown
Our PID: 18390
Owner Operation: unknown
Our operation:
Lock file: /home/jenkins/.gradle/daemon/6.8.3/registry.bin.lock
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PerformanceTests_ParquetIOIT
#4379
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_ParquetIOIT/4379/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_ParquetIOIT #4378
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_ParquetIOIT/4378/display/redirect?page=changes>
Changes:
[Brian Hulette] Add support std and var in DataFrame and agg
[zhoufek] [BEAM-9487] Raise error in GroupByKey on invalid PCollections
[Brian Hulette] Test std edge cases
[noreply] [BEAM-12199] Run pack_combiners phase by default in FnApiRunner and
[noreply] Merge pull request #14531 from [BEAM-12266] Adds a connector that
------------------------------------------
[...truncated 318.74 KB...]
May 18, 2021 6:27:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.428Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
May 18, 2021 6:27:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.462Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
May 18, 2021 6:27:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.498Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read
May 18, 2021 6:27:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.534Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.585Z: Fusing consumer Calculate hashcode/Values/Values/Map into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.603Z: Unzipping flatten s9-u129 for input s10.org.apache.beam.sdk.values.PCollection.<init>:402#2587af97b4865538-c127
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.629Z: Fusing unzipped copy of Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign, through flatten Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections/Unzipped-1, into producer Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.657Z: Fusing consumer Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.693Z: Fusing consumer Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign into Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.720Z: Fusing consumer Produce text lines into Generate sequence/Read(BoundedCountingSource)
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.782Z: Fusing consumer Produce Avro records into Produce text lines
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.816Z: Fusing consumer Gather write start times into Produce Avro records
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.849Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into Gather write start times
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.883Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.926Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:34.962Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.008Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.117Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.142Z: Fusing consumer PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) into Calculate hashcode/ProduceDefault
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.179Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map into Calculate hashcode/ProduceDefault
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.207Z: Fusing consumer Calculate hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.239Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/Reify into PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.278Z: Fusing consumer PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.715Z: Executing operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:35.794Z: Starting 5 ****s in us-central1-f...
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:36.392Z: Finished operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:36.581Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Create
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:36.732Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Create
May 18, 2021 6:27:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:36.909Z: Executing operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Produce Avro records+Gather write start times+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
May 18, 2021 6:27:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:27:53.778Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 18, 2021 6:28:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:28:23.327Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
May 18, 2021 6:28:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:28:49.197Z: Workers have started successfully.
May 18, 2021 6:28:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:28:49.228Z: Workers have started successfully.
May 18, 2021 6:31:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:55.361Z: Finished operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Produce Avro records+Gather write start times+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
May 18, 2021 6:31:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:55.433Z: Executing operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
May 18, 2021 6:31:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:55.591Z: Finished operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
May 18, 2021 6:31:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:55.702Z: Executing operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write
May 18, 2021 6:31:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:56.129Z: Finished operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write
May 18, 2021 6:31:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:56.203Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Close
May 18, 2021 6:31:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:56.257Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Close
May 18, 2021 6:31:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:56.342Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:31:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:56.518Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:31:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:56.682Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/GatherTempFileResults/Drop key/Values/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Gather bundles+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair with random key+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:31:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:58.266Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/GatherTempFileResults/Drop key/Values/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Gather bundles+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair with random key+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:31:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:58.352Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:31:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:58.418Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:31:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:58.554Z: Executing operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:32:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:58.762Z: Finished operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:32:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:31:58.939Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Values/Values/Map+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Finalize+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:32:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:02.952Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Values/Values/Map+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Finalize+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:32:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:03.046Z: Executing operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:32:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:04.025Z: Finished operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:32:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:04.113Z: Executing operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:32:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:04.281Z: Finished operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 6:32:06 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:04.428Z: Executing operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Gather write end times+Get file names/Values/Map+Find files/Match filepatterns+Find files/Reshuffle.ViaRandomKey/Pair with random key+Find files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:32:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:06.942Z: Finished operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Gather write end times+Get file names/Values/Map+Find files/Match filepatterns+Find files/Reshuffle.ViaRandomKey/Pair with random key+Find files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 6:32:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:07.053Z: Executing operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:32:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:07.106Z: Finished operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 6:32:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:07.173Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
May 18, 2021 6:32:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:07.326Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
May 18, 2021 6:32:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:32:07.484Z: Executing operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Find files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Find files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matched files/ParDo(ToReadableFile)+Gather read start time+Read parquet files/ParDo(Read)+Gather read end time+Map records to strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
May 18, 2021 6:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:44.637Z: Finished operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Find files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Find files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matched files/ParDo(ToReadableFile)+Gather read start time+Read parquet files/ParDo(Read)+Gather read end time+Map records to strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
May 18, 2021 6:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:44.733Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Close
May 18, 2021 6:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:44.798Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Close
May 18, 2021 6:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:44.949Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
May 18, 2021 6:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:44.994Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Create
May 18, 2021 6:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:45.109Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
May 18, 2021 6:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:45.138Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Create
May 18, 2021 6:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:45.414Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate hashcode/Values/Values/Map+Calculate hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 18, 2021 6:33:45 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:45.448Z: Executing operation PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 18, 2021 6:33:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:46.759Z: Finished operation PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 18, 2021 6:33:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:47.803Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate hashcode/Values/Values/Map+Calculate hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 18, 2021 6:33:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:47.951Z: Executing operation Calculate hashcode/View.AsIterable/CreateDataflowView
May 18, 2021 6:33:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:47.985Z: Finished operation Calculate hashcode/View.AsIterable/CreateDataflowView
May 18, 2021 6:33:48 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:48.147Z: Executing operation Calculate hashcode/CreateVoid/Read(CreateSource)+Calculate hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
May 18, 2021 6:33:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:48.865Z: Finished operation Calculate hashcode/CreateVoid/Read(CreateSource)+Calculate hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
May 18, 2021 6:33:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:48.944Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
May 18, 2021 6:33:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:48.976Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Close
May 18, 2021 6:33:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:49.019Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
May 18, 2021 6:33:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:49.036Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Close
May 18, 2021 6:33:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:49.089Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
May 18, 2021 6:33:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:49.125Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
May 18, 2021 6:33:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:49.258Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
May 18, 2021 6:33:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:49.396Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
May 18, 2021 6:33:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:49.932Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
May 18, 2021 6:33:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:50.005Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
May 18, 2021 6:33:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:50.086Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
May 18, 2021 6:33:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:50.177Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
May 18, 2021 6:33:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:51.006Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
May 18, 2021 6:33:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:51.580Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
May 18, 2021 6:33:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:51.745Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
May 18, 2021 6:33:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:51.791Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
May 18, 2021 6:33:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:51.962Z: Executing operation Delete test files
May 18, 2021 6:33:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:54.491Z: Finished operation Delete test files
May 18, 2021 6:33:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:54.660Z: Cleaning up.
May 18, 2021 6:33:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:33:54.737Z: Stopping **** pool...
May 18, 2021 6:34:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:34:46.146Z: Autoscaling: Resized **** pool from 5 to 0.
May 18, 2021 6:34:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T18:34:46.196Z: Worker pool stopped.
May 18, 2021 6:35:02 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-05-18_11_27_05-7998019210395361861 finished with status DONE.
org.apache.beam.sdk.io.parquet.ParquetIOIT > writeThenReadAll STANDARD_OUT
Load test results for test (ID): bffb27ab-7ef2-4977-be4f-0f30fdf2f456 and timestamp: 2021-05-18T18:35:03.235000000Z:
Metric: Value:
read_time 96.592
write_time 181.415
run_time 279.367
dataset_size 1.08737E9
org.apache.beam.sdk.io.parquet.ParquetIOIT > writeThenReadAll STANDARD_ERROR
log4j:WARN No appenders could be found for logger (org.apache.http.client.protocol.RequestAddCookies).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest
Finished generating test XML results (0.07 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_ParquetIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.083 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_ParquetIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
Invalidating in-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin
Stored cache entry for task ':sdks:java:io:file-based-io-tests:integrationTest' with cache key c6cdf9f1786007343d9f2cd2271eeee2
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Daemon ****,5,main]) completed. Took 10 mins 5.394 secs.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 11m 41s
120 actionable tasks: 73 executed, 47 from cache
Publishing build scan...
https://gradle.com/s/62zh5kgd5meii
FAILURE: Build failed with an exception.
* What went wrong:
Timeout waiting to lock daemon addresses registry. It is currently in use by another Gradle instance.
Owner PID: unknown
Our PID: 12980
Owner Operation: unknown
Our operation:
Lock file: /home/jenkins/.gradle/daemon/6.8.3/registry.bin.lock
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_ParquetIOIT #4377
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_ParquetIOIT/4377/display/redirect?page=changes>
Changes:
[noreply] Fix overview.md for cross-language pattern
------------------------------------------
[...truncated 317.20 KB...]
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:33.788Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:33.827Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:33.875Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:33.922Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:33.943Z: Fusing consumer Calculate hashcode/Values/Values/Map into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:33.967Z: Unzipping flatten s9-u129 for input s10.org.apache.beam.sdk.values.PCollection.<init>:402#2587af97b4865538-c127
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:33.991Z: Fusing unzipped copy of Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign, through flatten Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections/Unzipped-1, into producer Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:34.019Z: Fusing consumer Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:34.054Z: Fusing consumer Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign into Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:34.085Z: Fusing consumer Produce text lines into Generate sequence/Read(BoundedCountingSource)
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:34.120Z: Fusing consumer Produce Avro records into Produce text lines
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:34.163Z: Fusing consumer Gather write start times into Produce Avro records
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:34.201Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into Gather write start times
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:34.254Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:34.289Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:34.312Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:34.354Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:34.388Z: Fusing consumer Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:34.444Z: Fusing consumer PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) into Calculate hashcode/ProduceDefault
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:34.468Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map into Calculate hashcode/ProduceDefault
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:34.502Z: Fusing consumer Calculate hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:34.540Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/Reify into PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:34.572Z: Fusing consumer PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:35.208Z: Executing operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:35.284Z: Starting 5 ****s in us-central1-f...
May 18, 2021 12:25:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:35.652Z: Finished operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
May 18, 2021 12:25:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:35.800Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Create
May 18, 2021 12:25:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:36.004Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Create
May 18, 2021 12:25:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:36.156Z: Executing operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Produce Avro records+Gather write start times+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
May 18, 2021 12:25:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:25:58.539Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 18, 2021 12:26:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:26:20.625Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
May 18, 2021 12:26:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:26:49.845Z: Workers have started successfully.
May 18, 2021 12:26:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:26:49.880Z: Workers have started successfully.
May 18, 2021 12:30:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:08.144Z: Finished operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Produce Avro records+Gather write start times+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
May 18, 2021 12:30:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:08.203Z: Executing operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
May 18, 2021 12:30:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:08.274Z: Finished operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
May 18, 2021 12:30:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:08.491Z: Executing operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write
May 18, 2021 12:30:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:09.964Z: Finished operation Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write Parquet files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+Write Parquet files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write
May 18, 2021 12:30:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:10.052Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Close
May 18, 2021 12:30:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:10.111Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Close
May 18, 2021 12:30:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:10.177Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 12:30:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:10.373Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 12:30:10 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:10.519Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/GatherTempFileResults/Drop key/Values/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Gather bundles+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair with random key+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 12:30:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:13.395Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/GatherTempFileResults/Drop key/Values/Map+Write Parquet files/WriteFiles/GatherTempFileResults/Gather bundles+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair with random key+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 12:30:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:13.465Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 12:30:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:13.527Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 12:30:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:13.593Z: Executing operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 12:30:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:13.789Z: Finished operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 12:30:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:13.969Z: Executing operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Values/Values/Map+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Finalize+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 12:30:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:15.552Z: Finished operation Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Values/Values/Map+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Finalize+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 12:30:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:15.616Z: Executing operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 12:30:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:15.667Z: Finished operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 12:30:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:15.754Z: Executing operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 12:30:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:15.928Z: Finished operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 18, 2021 12:30:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:16.082Z: Executing operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Gather write end times+Get file names/Values/Map+Find files/Match filepatterns+Find files/Reshuffle.ViaRandomKey/Pair with random key+Find files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 12:30:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:19.299Z: Finished operation Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write Parquet files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Gather write end times+Get file names/Values/Map+Find files/Match filepatterns+Find files/Reshuffle.ViaRandomKey/Pair with random key+Find files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 18, 2021 12:30:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:19.404Z: Executing operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 12:30:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:19.458Z: Finished operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 18, 2021 12:30:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:19.524Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
May 18, 2021 12:30:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:19.997Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
May 18, 2021 12:30:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:30:20.118Z: Executing operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Find files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Find files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matched files/ParDo(ToReadableFile)+Gather read start time+Read parquet files/ParDo(Read)+Gather read end time+Map records to strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
May 18, 2021 12:32:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:26.465Z: Finished operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Find files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Find files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matched files/ParDo(ToReadableFile)+Gather read start time+Read parquet files/ParDo(Read)+Gather read end time+Map records to strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
May 18, 2021 12:32:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:26.542Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Close
May 18, 2021 12:32:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:26.601Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Close
May 18, 2021 12:32:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:26.672Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
May 18, 2021 12:32:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:26.768Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Create
May 18, 2021 12:32:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:26.844Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
May 18, 2021 12:32:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:26.942Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Create
May 18, 2021 12:32:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:27.102Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate hashcode/Values/Values/Map+Calculate hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 18, 2021 12:32:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:27.145Z: Executing operation PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 18, 2021 12:32:29 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:28.443Z: Finished operation PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 18, 2021 12:32:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:30.810Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate hashcode/Values/Values/Map+Calculate hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 18, 2021 12:32:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:30.986Z: Executing operation Calculate hashcode/View.AsIterable/CreateDataflowView
May 18, 2021 12:32:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:31.035Z: Finished operation Calculate hashcode/View.AsIterable/CreateDataflowView
May 18, 2021 12:32:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:31.202Z: Executing operation Calculate hashcode/CreateVoid/Read(CreateSource)+Calculate hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
May 18, 2021 12:32:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:31.883Z: Finished operation Calculate hashcode/CreateVoid/Read(CreateSource)+Calculate hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
May 18, 2021 12:32:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:31.971Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
May 18, 2021 12:32:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:32.021Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Close
May 18, 2021 12:32:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:32.061Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
May 18, 2021 12:32:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:32.079Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Close
May 18, 2021 12:32:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:32.130Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
May 18, 2021 12:32:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:35.785Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
May 18, 2021 12:32:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:35.968Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
May 18, 2021 12:32:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:36.102Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
May 18, 2021 12:32:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:37.491Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
May 18, 2021 12:32:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:37.562Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
May 18, 2021 12:32:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:37.609Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
May 18, 2021 12:32:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:37.676Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
May 18, 2021 12:32:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:38.229Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
May 18, 2021 12:32:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:39.062Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
May 18, 2021 12:32:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:39.197Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
May 18, 2021 12:32:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:39.248Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
May 18, 2021 12:32:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:39.377Z: Executing operation Delete test files
May 18, 2021 12:32:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:43.556Z: Finished operation Delete test files
May 18, 2021 12:32:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:43.738Z: Cleaning up.
May 18, 2021 12:32:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:32:43.816Z: Stopping **** pool...
May 18, 2021 12:33:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:33:38.274Z: Autoscaling: Resized **** pool from 5 to 0.
May 18, 2021 12:33:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-18T12:33:38.316Z: Worker pool stopped.
May 18, 2021 12:33:54 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-05-18_05_25_13-2273784061106767290 finished with status DONE.
org.apache.beam.sdk.io.parquet.ParquetIOIT > writeThenReadAll STANDARD_OUT
Load test results for test (ID): a497e1ee-9478-4801-8b3a-827b44d31136 and timestamp: 2021-05-18T12:33:54.390000000Z:
Metric: Value:
run_time 319.049
write_time 192.191
read_time 124.944
dataset_size 1.08737E9
org.apache.beam.sdk.io.parquet.ParquetIOIT > writeThenReadAll STANDARD_ERROR
log4j:WARN No appenders could be found for logger (org.apache.http.client.protocol.RequestAddCookies).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest
Finished generating test XML results (0.034 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_ParquetIOIT/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.055 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_ParquetIOIT/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
Invalidating in-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin
Stored cache entry for task ':sdks:java:io:file-based-io-tests:integrationTest' with cache key c6cdf9f1786007343d9f2cd2271eeee2
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 11,5,main]) completed. Took 9 mins 13.766 secs.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 10m 22s
120 actionable tasks: 73 executed, 47 from cache
Publishing build scan...
https://gradle.com/s/ar4uyovgb2256
FAILURE: Build failed with an exception.
* What went wrong:
Timeout waiting to lock daemon addresses registry. It is currently in use by another Gradle instance.
Owner PID: unknown
Our PID: 10206
Owner Operation: unknown
Our operation:
Lock file: /home/jenkins/.gradle/daemon/6.8.3/registry.bin.lock
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org