You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2020/10/14 01:02:44 UTC
Build failed in Jenkins:
beam_PerformanceTests_ManyFiles_TextIOIT_HDFS #2600
See <https://ci-beam.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT_HDFS/2600/display/redirect?page=changes>
Changes:
[rohde.samuel] Add ability for HotKeyLogger to log a key.
[noreply] [BEAM-10861]Adds PubSub Runner API encoding to Read/Write transforms
[noreply] [BEAM-8251] Add worker_region and worker_zone options (#13072)
[noreply] [BEAM-10720] Finish implementing StringMethods (cat, repeat) (#12705)
[noreply] [BEAM-10009] Document micros_instant (#12921)
[noreply] [BEAM-10987] Skip test_with_main_session on windows (#13099)
------------------------------------------
[...truncated 310.05 KB...]
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:14.944Z: Fusing consumer Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign into Read files/Read all via FileBasedSource/Reshuffle/Pair with random key
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.022Z: Fusing consumer Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify into Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.050Z: Fusing consumer Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Write into Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.081Z: Fusing consumer Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow into Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Read
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.105Z: Fusing consumer Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/ExpandIterable into Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.140Z: Fusing consumer Read files/Read all via FileBasedSource/Reshuffle/Values/Values/Map into Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/ExpandIterable
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.166Z: Fusing consumer Read files/Read all via FileBasedSource/Read ranges into Read files/Read all via FileBasedSource/Reshuffle/Values/Values/Map
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.201Z: Fusing consumer Collect read end time into Read files/Read all via FileBasedSource/Read ranges
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.236Z: Fusing consumer Calculate hashcode/WithKeys/AddKeys/Map into Collect read end time
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.261Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate hashcode/WithKeys/AddKeys/Map
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.297Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.333Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.368Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.404Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.439Z: Fusing consumer Calculate hashcode/Values/Values/Map into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.477Z: Fusing consumer Calculate hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.510Z: Fusing consumer PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Oct 14, 2020 12:58:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.534Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/Reify into PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.931Z: Executing operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:15.966Z: Executing operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:16.001Z: Executing operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Create
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:16.001Z: Starting 5 ****s in us-central1-c...
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:16.037Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:16.047Z: Finished operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:16.063Z: Finished operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Create
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:16.063Z: Finished operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:16.080Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Create
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:16.094Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:16.109Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:16.131Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Create
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:16.143Z: Executing operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Create
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:16.167Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:16.180Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:16.205Z: Finished operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Create
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:16.246Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:16.512Z: Executing operation PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
Oct 14, 2020 12:58:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:16.548Z: Executing operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Collect write start time+Write content to files/WriteFiles/RewindowIntoGlobal/Window.Assign+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/ApplyShardingKey+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Reify+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Write
Oct 14, 2020 12:58:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:33.035Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 14, 2020 12:58:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:58:41.236Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
Oct 14, 2020 12:59:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:59:00.683Z: Workers have started successfully.
Oct 14, 2020 12:59:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:59:00.736Z: Workers have started successfully.
Oct 14, 2020 12:59:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T00:59:42.853Z: Finished operation PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
Oct 14, 2020 1:00:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:00:02.180Z: Finished operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Collect write start time+Write content to files/WriteFiles/RewindowIntoGlobal/Window.Assign+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/ApplyShardingKey+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Reify+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Write
Oct 14, 2020 1:00:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:00:02.263Z: Executing operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Close
Oct 14, 2020 1:00:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:00:02.329Z: Finished operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Close
Oct 14, 2020 1:00:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:00:02.411Z: Executing operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Read+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/GroupByWindow+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/WriteShardsIntoTempFiles+Write content to files/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
Oct 14, 2020 1:01:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:14.226Z: Finished operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Read+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/GroupByWindow+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/WriteShardsIntoTempFiles+Write content to files/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
Oct 14, 2020 1:01:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:14.383Z: Executing operation Write content to files/WriteFiles/GatherTempFileResults/View.AsList/CreateDataflowView
Oct 14, 2020 1:01:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:14.475Z: Finished operation Write content to files/WriteFiles/GatherTempFileResults/View.AsList/CreateDataflowView
Oct 14, 2020 1:01:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:14.642Z: Executing operation Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+Write content to files/WriteFiles/FinalizeTempFileBundles/Finalize+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Oct 14, 2020 1:01:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:22.936Z: Finished operation Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+Write content to files/WriteFiles/FinalizeTempFileBundles/Finalize+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Oct 14, 2020 1:01:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:23.207Z: Executing operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Oct 14, 2020 1:01:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:23.336Z: Finished operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Oct 14, 2020 1:01:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:23.406Z: Executing operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Values/Values/Map+Collect write end time+Match all files/Match filepatterns+Match all files/Reshuffle.ViaRandomKey/Pair with random key+Match all files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Oct 14, 2020 1:01:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:26.723Z: Finished operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Values/Values/Map+Collect write end time+Match all files/Match filepatterns+Match all files/Reshuffle.ViaRandomKey/Pair with random key+Match all files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Oct 14, 2020 1:01:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:27.123Z: Executing operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Oct 14, 2020 1:01:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:27.188Z: Finished operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Oct 14, 2020 1:01:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:27.287Z: Executing operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Match all files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Match all files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matches/ParDo(ToReadableFile)+Read files/Read all via FileBasedSource/Split into ranges+Read files/Read all via FileBasedSource/Reshuffle/Pair with random key+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Write
Oct 14, 2020 1:01:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:28.916Z: Finished operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Match all files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Match all files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matches/ParDo(ToReadableFile)+Read files/Read all via FileBasedSource/Split into ranges+Read files/Read all via FileBasedSource/Reshuffle/Pair with random key+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Write
Oct 14, 2020 1:01:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:28.986Z: Executing operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Close
Oct 14, 2020 1:01:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:29.198Z: Finished operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Close
Oct 14, 2020 1:01:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:29.411Z: Executing operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Read+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/ExpandIterable+Read files/Read all via FileBasedSource/Reshuffle/Values/Values/Map+Read files/Read all via FileBasedSource/Read ranges+Collect read end time+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Oct 14, 2020 1:01:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:49.150Z: Finished operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Read+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/ExpandIterable+Read files/Read all via FileBasedSource/Reshuffle/Values/Values/Map+Read files/Read all via FileBasedSource/Read ranges+Collect read end time+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Oct 14, 2020 1:01:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:49.255Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Close
Oct 14, 2020 1:01:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:49.320Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Close
Oct 14, 2020 1:01:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:49.389Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate hashcode/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+Calculate hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)
Oct 14, 2020 1:01:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:51.291Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate hashcode/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+Calculate hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)
Oct 14, 2020 1:01:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:51.473Z: Executing operation Calculate hashcode/View.AsIterable/CreateDataflowView
Oct 14, 2020 1:01:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:51.526Z: Finished operation Calculate hashcode/View.AsIterable/CreateDataflowView
Oct 14, 2020 1:01:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:51.693Z: Executing operation Calculate hashcode/CreateVoid/Read(CreateSource)+Calculate hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
Oct 14, 2020 1:01:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:52.550Z: Finished operation Calculate hashcode/CreateVoid/Read(CreateSource)+Calculate hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
Oct 14, 2020 1:01:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:52.628Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Oct 14, 2020 1:01:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:52.663Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Close
Oct 14, 2020 1:01:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:52.679Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Oct 14, 2020 1:01:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:52.728Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Close
Oct 14, 2020 1:01:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:52.753Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Oct 14, 2020 1:01:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:52.802Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
Oct 14, 2020 1:01:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:53.378Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
Oct 14, 2020 1:01:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:53.423Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Oct 14, 2020 1:01:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:53.509Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Oct 14, 2020 1:01:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:53.565Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Oct 14, 2020 1:01:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:53.637Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Oct 14, 2020 1:01:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:55.574Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Oct 14, 2020 1:01:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:55.718Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
Oct 14, 2020 1:01:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:55.771Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
Oct 14, 2020 1:01:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:55.937Z: Executing operation Delete test files
Oct 14, 2020 1:02:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:59.007Z: Finished operation Delete test files
Oct 14, 2020 1:02:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:59.175Z: Cleaning up.
Oct 14, 2020 1:02:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-14T01:01:59.255Z: Stopping **** pool...
Oct 14, 2020 1:02:43 AM org.apache.beam.runners.dataflow.DataflowPipelineJob lambda$waitUntilFinish$0
WARNING: Job is already running in Google Cloud Platform, Ctrl-C will not cancel it.
To cancel the job in the cloud, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-10-13_17_57_53-16364477530988434267
org.apache.beam.sdk.io.text.TextIOIT > writeThenReadAll SKIPPED
> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 5 mins 4.063 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> Process 'Gradle Test Executor 6' finished with non-zero exit value 143
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at https://docs.gradle.org/6.6.1/userguide/java_testing.html#sec:test_execution
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 48s
96 actionable tasks: 59 executed, 37 from cache
Publishing build scan...
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=332bc08b-412f-436e-a7e4-a41976f66b90, currentDir=<https://ci-beam.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT_HDFS/ws/src}>
Attempting to read last messages from the daemon log...
Daemon pid: 20274
log file: /home/jenkins/.gradle/daemon/6.6.1/daemon-20274.out.log
----- Last 20 lines from daemon log file - daemon-20274.out.log -----
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> Process 'Gradle Test Executor 6' finished with non-zero exit value 143
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at https://docs.gradle.org/6.6.1/userguide/java_testing.html#sec:test_execution
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 48s
96 actionable tasks: 59 executed, 37 from cache
Publishing build scan...
Daemon vm is shutting down... The daemon has exited normally or was terminated in response to a user interrupt.
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_PerformanceTests_ManyFiles_TextIOIT_HDFS #2601
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT_HDFS/2601/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org