You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/05/19 06:27:56 UTC
Build failed in Jenkins: beam_PerformanceTests_XmlIOIT_HDFS #4541
See <https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/4541/display/redirect>
Changes:
------------------------------------------
[...truncated 341.48 KB...]
INFO: 2021-05-19T06:22:52.143Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.165Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.199Z: Fusing consumer Calculate hashcode/Values/Values/Map into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.238Z: Unzipping flatten s8-u138 for input s9.org.apache.beam.sdk.values.PCollection.<init>:402#20ff67585e33a8f6-c136
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.270Z: Fusing unzipped copy of Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign, through flatten Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections/Unzipped-1, into producer Write xml files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.304Z: Fusing consumer Write xml files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map into Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.328Z: Fusing consumer Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign into Write xml files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.350Z: Fusing consumer PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) into Calculate hashcode/ProduceDefault
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.372Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map into Calculate hashcode/ProduceDefault
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.415Z: Fusing consumer Create xml records/Map into Generate sequence/Read(BoundedCountingSource)
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.442Z: Fusing consumer Gather write start time into Create xml records/Map
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.464Z: Fusing consumer Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into Gather write start time
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.486Z: Fusing consumer Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.546Z: Fusing consumer Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.571Z: Fusing consumer Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.592Z: Fusing consumer Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.625Z: Fusing consumer Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.647Z: Fusing consumer Calculate hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.670Z: Fusing consumer PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:52.696Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/Reify into PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:53.234Z: Executing operation Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:53.301Z: Starting 5 ****s in us-central1-c...
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:53.683Z: Finished operation Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:53.841Z: Executing operation Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Create
May 19, 2021 6:22:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:54.006Z: Finished operation Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Create
May 19, 2021 6:22:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:22:54.170Z: Executing operation Generate sequence/Read(BoundedCountingSource)+Create xml records/Map+Gather write start time+Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write xml files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write+Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
May 19, 2021 6:23:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:23:12.203Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 19, 2021 6:23:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:23:36.827Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
May 19, 2021 6:24:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:02.963Z: Workers have started successfully.
May 19, 2021 6:24:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:02.993Z: Workers have started successfully.
May 19, 2021 6:24:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:48.027Z: Finished operation Generate sequence/Read(BoundedCountingSource)+Create xml records/Map+Gather write start time+Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write xml files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write+Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
May 19, 2021 6:24:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:48.124Z: Executing operation Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
May 19, 2021 6:24:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:48.183Z: Finished operation Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
May 19, 2021 6:24:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:48.249Z: Executing operation Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+Write xml files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write
May 19, 2021 6:24:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:48.676Z: Finished operation Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+Write xml files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write
May 19, 2021 6:24:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:48.738Z: Executing operation Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Close
May 19, 2021 6:24:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:48.795Z: Finished operation Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Close
May 19, 2021 6:24:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:48.855Z: Executing operation Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 19, 2021 6:24:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:49.102Z: Finished operation Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 19, 2021 6:24:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:49.216Z: Executing operation Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Read+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/GroupByWindow+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable+Write xml files/WriteFiles/GatherTempFileResults/Drop key/Values/Map+Write xml files/WriteFiles/GatherTempFileResults/Gather bundles+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair with random key+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 19, 2021 6:24:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:52.411Z: Finished operation Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Read+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/GroupByWindow+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable+Write xml files/WriteFiles/GatherTempFileResults/Drop key/Values/Map+Write xml files/WriteFiles/GatherTempFileResults/Gather bundles+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair with random key+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 19, 2021 6:24:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:52.487Z: Executing operation Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 19, 2021 6:24:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:52.541Z: Finished operation Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 19, 2021 6:24:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:52.611Z: Executing operation Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 19, 2021 6:24:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:52.800Z: Finished operation Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 19, 2021 6:24:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:52.945Z: Executing operation Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Values/Values/Map+Write xml files/WriteFiles/FinalizeTempFileBundles/Finalize+Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 19, 2021 6:24:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:54.816Z: Finished operation Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write xml files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Values/Values/Map+Write xml files/WriteFiles/FinalizeTempFileBundles/Finalize+Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 19, 2021 6:24:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:54.887Z: Executing operation Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 19, 2021 6:24:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:54.943Z: Finished operation Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 19, 2021 6:24:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:54.995Z: Executing operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 19, 2021 6:24:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:55.420Z: Finished operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 19, 2021 6:24:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:55.549Z: Executing operation Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Gather write end time+Get file names/Values/Map+Find files/Match filepatterns+Find files/Reshuffle.ViaRandomKey/Pair with random key+Find files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 19, 2021 6:25:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:59.822Z: Finished operation Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write xml files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Gather write end time+Get file names/Values/Map+Find files/Match filepatterns+Find files/Reshuffle.ViaRandomKey/Pair with random key+Find files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 19, 2021 6:25:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:59.928Z: Executing operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 19, 2021 6:25:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:24:59.974Z: Finished operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 19, 2021 6:25:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:00.056Z: Executing operation Read xml files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/GroupByKey/Create
May 19, 2021 6:25:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:00.262Z: Finished operation Read xml files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/GroupByKey/Create
May 19, 2021 6:25:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:00.396Z: Executing operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Find files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Find files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matched files/ParDo(ToReadableFile)+Gather read start time+Read xml files/ReadAllViaFileBasedSource/Split into ranges+Read xml files/ReadAllViaFileBasedSource/Reshuffle/Pair with random key+Read xml files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read xml files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify+Read xml files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/GroupByKey/Write
May 19, 2021 6:25:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:01.050Z: Finished operation Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Find files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Find files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Find files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matched files/ParDo(ToReadableFile)+Gather read start time+Read xml files/ReadAllViaFileBasedSource/Split into ranges+Read xml files/ReadAllViaFileBasedSource/Reshuffle/Pair with random key+Read xml files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read xml files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify+Read xml files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/GroupByKey/Write
May 19, 2021 6:25:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:01.135Z: Executing operation Read xml files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/GroupByKey/Close
May 19, 2021 6:25:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:01.203Z: Finished operation Read xml files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/GroupByKey/Close
May 19, 2021 6:25:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:01.260Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
May 19, 2021 6:25:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:01.461Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
May 19, 2021 6:25:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:01.605Z: Executing operation Read xml files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/GroupByKey/Read+Read xml files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+Read xml files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/ExpandIterable+Read xml files/ReadAllViaFileBasedSource/Reshuffle/Values/Values/Map+Read xml files/ReadAllViaFileBasedSource/Read ranges+Gather read end time+Map xml records to strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
May 19, 2021 6:25:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:28.960Z: Finished operation Read xml files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/GroupByKey/Read+Read xml files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+Read xml files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/ExpandIterable+Read xml files/ReadAllViaFileBasedSource/Reshuffle/Values/Values/Map+Read xml files/ReadAllViaFileBasedSource/Read ranges+Gather read end time+Map xml records to strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
May 19, 2021 6:25:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:29.026Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Close
May 19, 2021 6:25:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:30.089Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Close
May 19, 2021 6:25:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:30.189Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
May 19, 2021 6:25:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:30.235Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Create
May 19, 2021 6:25:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:30.428Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
May 19, 2021 6:25:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:30.435Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Create
May 19, 2021 6:25:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:30.598Z: Executing operation PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 19, 2021 6:25:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:30.632Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate hashcode/Values/Values/Map+Calculate hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 19, 2021 6:25:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:30.853Z: Finished operation PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 19, 2021 6:25:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:32.409Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate hashcode/Values/Values/Map+Calculate hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 19, 2021 6:25:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:32.601Z: Executing operation Calculate hashcode/View.AsIterable/CreateDataflowView
May 19, 2021 6:25:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:32.647Z: Finished operation Calculate hashcode/View.AsIterable/CreateDataflowView
May 19, 2021 6:25:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:32.777Z: Executing operation Calculate hashcode/CreateVoid/Read(CreateSource)+Calculate hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
May 19, 2021 6:25:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:33.385Z: Finished operation Calculate hashcode/CreateVoid/Read(CreateSource)+Calculate hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
May 19, 2021 6:25:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:33.451Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
May 19, 2021 6:25:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:33.477Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Close
May 19, 2021 6:25:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:33.500Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
May 19, 2021 6:25:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:33.531Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Close
May 19, 2021 6:25:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:33.564Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
May 19, 2021 6:25:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:33.609Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
May 19, 2021 6:25:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:33.713Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
May 19, 2021 6:25:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:33.868Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
May 19, 2021 6:25:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:34.220Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
May 19, 2021 6:25:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:34.348Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
May 19, 2021 6:25:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:34.424Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
May 19, 2021 6:25:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:34.471Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
May 19, 2021 6:25:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:34.544Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
May 19, 2021 6:25:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:36.408Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
May 19, 2021 6:25:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:36.570Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
May 19, 2021 6:25:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:36.616Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
May 19, 2021 6:25:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:36.761Z: Executing operation Delete test files
May 19, 2021 6:25:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:38.362Z: Finished operation Delete test files
May 19, 2021 6:25:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:38.558Z: Cleaning up.
May 19, 2021 6:25:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:25:38.624Z: Stopping **** pool...
May 19, 2021 6:26:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:26:36.437Z: Autoscaling: Resized **** pool from 5 to 0.
May 19, 2021 6:26:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-19T06:26:36.496Z: Worker pool stopped.
May 19, 2021 6:26:42 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-05-18_23_22_27-1273931473004215694 finished with status DONE.
org.apache.beam.sdk.io.xml.XmlIOIT > writeThenReadAll STANDARD_OUT
Load test results for test (ID): 09d3bd92-ad84-4344-9f6e-f103ad095fbc and timestamp: 2021-05-19T06:26:43.060000000Z:
Metric: Value:
write_time 38.915
dataset_size 1.07659E9
read_time 28.15
run_time 69.138
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest
Finished generating test XML results (0.058 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.081 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
Invalidating in-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin
Stored cache entry for task ':sdks:java:io:file-based-io-tests:integrationTest' with cache key 1b952ba32649ea2ddfdff600eabf4802
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 5,5,main]) completed. Took 4 mins 57.822 secs.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 6m 19s
125 actionable tasks: 76 executed, 49 from cache
Publishing build scan...
https://gradle.com/s/ntbsncw5zyimk
FAILURE: Build failed with an exception.
* What went wrong:
Timeout waiting to lock daemon addresses registry. It is currently in use by another Gradle instance.
Owner PID: unknown
Our PID: 21194
Owner Operation: unknown
Our operation:
Lock file: /home/jenkins/.gradle/daemon/6.8.3/registry.bin.lock
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_PerformanceTests_XmlIOIT_HDFS #4543
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/4543/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_XmlIOIT_HDFS #4542
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/4542/display/redirect>
Changes:
------------------------------------------
Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on apache-beam-jenkins-5 (beam) in workspace <https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/>
The recommended git tool is: NONE
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/apache/beam.git
> git init <https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src> # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git --version # 'git version 2.7.4'
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* # timeout=10
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 1eaa8694528f44c8254992ad30da98aa8fb999cf (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 1eaa8694528f44c8254992ad30da98aa8fb999cf # timeout=10
Commit message: "[BEAM-7372][BEAM-9372] Cleanup py2 and py35 codepath from runners worker (#14730)"
> git rev-list --no-walk 1eaa8694528f44c8254992ad30da98aa8fb999cf # timeout=10
No emails were triggered.
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
SETUPTOOLS_USE_DISTUTILS=stdlib
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins1507804489202801391.sh
+ cp /home/jenkins/.kube/config <https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542>
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties content
KUBECONFIG=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542>
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins8077575868458293684.sh
+ gcloud container clusters get-credentials io-datastores --zone=us-central1-a
Fetching cluster endpoint and auth data.
kubeconfig entry generated for io-datastores.
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins7125258618781739028.sh
+ <https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/.test-infra/kubernetes/kubernetes.sh> createNamespace beam-performancetests-xmlioit-hdfs-4542
+ KUBECONFIG=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542>
+ KUBERNETES_NAMESPACE=default
+ KUBECTL='kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=default'
+ createNamespace beam-performancetests-xmlioit-hdfs-4542
+ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> create namespace beam-performancetests-xmlioit-hdfs-4542'
++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> create namespace beam-performancetests-xmlioit-hdfs-4542
namespace/beam-performancetests-xmlioit-hdfs-4542 created
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties content
KUBERNETES_NAMESPACE=beam-performancetests-xmlioit-hdfs-4542
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins5772598590995773631.sh
+ <https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/.test-infra/kubernetes/kubernetes.sh> apply <https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/LargeITCluster/hdfs-multi-datanode-cluster.yml>
+ KUBECONFIG=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542>
+ KUBERNETES_NAMESPACE=beam-performancetests-xmlioit-hdfs-4542
+ KUBECTL='kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542'
+ apply <https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/LargeITCluster/hdfs-multi-datanode-cluster.yml>
+ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542 apply -R -f <https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/LargeITCluster/hdfs-multi-datanode-cluster.yml'>
++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542 apply -R -f <https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/LargeITCluster/hdfs-multi-datanode-cluster.yml>
service/hadoop created
service/hadoop-datanodes created
statefulset.apps/datanode created
pod/namenode-0 created
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins5062274008661242272.sh
+ set -eo pipefail
+ sed 's/^/LOAD_BALANCER_IP=/'
+ eval <https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/.test-infra/kubernetes/kubernetes.sh> loadBalancerIP hadoop
++ <https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/.test-infra/kubernetes/kubernetes.sh> loadBalancerIP hadoop
+ KUBECONFIG=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542>
+ KUBERNETES_NAMESPACE=beam-performancetests-xmlioit-hdfs-4542
+ KUBECTL='kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542'
+ loadBalancerIP hadoop
+ local name=hadoop
+ local 'command=kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542 get svc hadoop -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\'''
+ retry 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542 get svc hadoop -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' 36 10
+ local 'command=kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542 get svc hadoop -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\'''
+ local max_retries=36
+ local sleep_time=10
+ (( i = 1 ))
+ (( i <= max_retries ))
+ local output
++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542 get svc hadoop -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\'''
+++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542 get svc hadoop '-ojsonpath={.status.loadBalancer.ingress[0].ip}'
+ output=
+ local status=0
+ [[ 0 == 0 ]]
+ [[ -n '' ]]
+ [[ 1 == \3\6 ]]
+ sleep 10
+ (( i++ ))
+ (( i <= max_retries ))
+ local output
++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542 get svc hadoop -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\'''
+++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542 get svc hadoop '-ojsonpath={.status.loadBalancer.ingress[0].ip}'
+ output=
+ local status=0
+ [[ 0 == 0 ]]
+ [[ -n '' ]]
+ [[ 2 == \3\6 ]]
+ sleep 10
+ (( i++ ))
+ (( i <= max_retries ))
+ local output
++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542 get svc hadoop -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\'''
+++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542 get svc hadoop '-ojsonpath={.status.loadBalancer.ingress[0].ip}'
+ output=
+ local status=0
+ [[ 0 == 0 ]]
+ [[ -n '' ]]
+ [[ 3 == \3\6 ]]
+ sleep 10
+ (( i++ ))
+ (( i <= max_retries ))
+ local output
++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542 get svc hadoop -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\'''
+++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542 get svc hadoop '-ojsonpath={.status.loadBalancer.ingress[0].ip}'
+ output=
+ local status=0
+ [[ 0 == 0 ]]
+ [[ -n '' ]]
+ [[ 4 == \3\6 ]]
+ sleep 10
+ (( i++ ))
+ (( i <= max_retries ))
+ local output
++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542 get svc hadoop -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\'''
+++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542 get svc hadoop '-ojsonpath={.status.loadBalancer.ingress[0].ip}'
+ output=
+ local status=0
+ [[ 0 == 0 ]]
+ [[ -n '' ]]
+ [[ 5 == \3\6 ]]
+ sleep 10
+ (( i++ ))
+ (( i <= max_retries ))
+ local output
++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542 get svc hadoop -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\'''
+++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-beam-performancetests-xmlioit-hdfs-4542> --namespace=beam-performancetests-xmlioit-hdfs-4542 get svc hadoop '-ojsonpath={.status.loadBalancer.ingress[0].ip}'
+ output=35.226.172.35
+ local status=0
+ [[ 0 == 0 ]]
+ [[ -n 35.226.172.35 ]]
+ echo 35.226.172.35
+ return 0
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties file path 'job.properties'
[EnvInject] - Variables injected successfully.
[Gradle] - Launching build.
[src] $ <https://ci-beam.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/gradlew> --continue --max-****s=12 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g -Dorg.gradle.vfs.watch=false -Pdocker-pull-licenses --info -DintegrationTestPipelineOptions=["--bigQueryDataset=beam_performance","--bigQueryTable=xmlioit_hdfs_results","--influxMeasurement=xmlioit_hdfs_results","--numberOfRecords=12000000","--expectedHash=b3b717e7df8f4878301b20f314512fb3","--datasetSize=1076590000","--charset=UTF-8","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--hdfsConfiguration=[{\"fs.defaultFS\":\"hdfs:35.226.172.35:9000\",\"dfs.replication\":1}]","--filenamePrefix=hdfs://35.226.172.35:9000/TEXTIO_IT_"] -Dfilesystem=hdfs -DintegrationTestRunner=dataflow :sdks:java:io:file-based-io-tests:integrationTest --tests org.apache.beam.sdk.io.xml.XmlIOIT
Initialized native services in: /home/jenkins/.gradle/native
Removing 0 daemon stop events from registry
FAILURE: Build failed with an exception.
* What went wrong:
Timeout waiting to lock daemon addresses registry. It is currently in use by another Gradle instance.
Owner PID: unknown
Our PID: 9131
Owner Operation: unknown
Our operation:
Lock file: /home/jenkins/.gradle/daemon/6.8.3/registry.bin.lock
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org