You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/05/07 07:02:50 UTC
Build failed in Jenkins:
beam_PerformanceTests_ManyFiles_TextIOIT_HDFS #3421
See <https://ci-beam.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT_HDFS/3421/display/redirect?page=changes>
Changes:
[Brian Hulette] Disable filesystem watching in Jenkins
[Kenneth Knowles] [BEAM-12250] Increase Dataflow ValidatesRunner timeouts
------------------------------------------
[...truncated 339.78 KB...]
May 07, 2021 6:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:14.751Z: Fusing consumer Read files/Read all via FileBasedSource/Reshuffle/Values/Values/Map into Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/ExpandIterable
May 07, 2021 6:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:14.777Z: Fusing consumer Read files/Read all via FileBasedSource/Read ranges into Read files/Read all via FileBasedSource/Reshuffle/Values/Values/Map
May 07, 2021 6:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:14.808Z: Fusing consumer Collect read end time into Read files/Read all via FileBasedSource/Read ranges
May 07, 2021 6:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:14.832Z: Fusing consumer Calculate hashcode/WithKeys/AddKeys/Map into Collect read end time
May 07, 2021 6:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:14.865Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate hashcode/WithKeys/AddKeys/Map
May 07, 2021 6:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:14.896Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
May 07, 2021 6:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:14.924Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
May 07, 2021 6:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:14.959Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read
May 07, 2021 6:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:15.002Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
May 07, 2021 6:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:15.036Z: Fusing consumer Calculate hashcode/Values/Values/Map into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
May 07, 2021 6:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:15.064Z: Fusing consumer Calculate hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
May 07, 2021 6:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:15.091Z: Fusing consumer PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
May 07, 2021 6:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:15.126Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/Reify into PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
May 07, 2021 6:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:15.479Z: Executing operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Create
May 07, 2021 6:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:15.547Z: Starting 5 ****s in us-central1-c...
May 07, 2021 6:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:15.896Z: Finished operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Create
May 07, 2021 6:58:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:16.061Z: Executing operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Collect write start time+Write content to files/WriteFiles/RewindowIntoGlobal/Window.Assign+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/ApplyShardingKey+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Reify+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Write
May 07, 2021 6:58:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:20.301Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 07, 2021 6:58:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:58:55.500Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
May 07, 2021 6:59:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:59:20.832Z: Workers have started successfully.
May 07, 2021 6:59:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T06:59:20.858Z: Workers have started successfully.
May 07, 2021 7:00:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:00:13.311Z: Finished operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Collect write start time+Write content to files/WriteFiles/RewindowIntoGlobal/Window.Assign+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/ApplyShardingKey+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Reify+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Write
May 07, 2021 7:00:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:00:13.381Z: Executing operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Close
May 07, 2021 7:00:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:00:14.386Z: Finished operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Close
May 07, 2021 7:00:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:00:14.457Z: Executing operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Read+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/GroupByWindow+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/WriteShardsIntoTempFiles+Write content to files/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
May 07, 2021 7:01:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:02.146Z: Finished operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Read+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/GroupByWindow+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/WriteShardsIntoTempFiles+Write content to files/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
May 07, 2021 7:01:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:03.399Z: Executing operation Write content to files/WriteFiles/GatherTempFileResults/View.AsList/CreateDataflowView
May 07, 2021 7:01:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:03.454Z: Finished operation Write content to files/WriteFiles/GatherTempFileResults/View.AsList/CreateDataflowView
May 07, 2021 7:01:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:03.586Z: Executing operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 07, 2021 7:01:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:03.883Z: Finished operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 07, 2021 7:01:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:04.295Z: Executing operation Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+Write content to files/WriteFiles/FinalizeTempFileBundles/Finalize+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 07, 2021 7:01:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:13.462Z: Finished operation Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+Write content to files/WriteFiles/FinalizeTempFileBundles/Finalize+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 07, 2021 7:01:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:13.679Z: Executing operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 07, 2021 7:01:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:15.034Z: Finished operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 07, 2021 7:01:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:15.092Z: Executing operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 07, 2021 7:01:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:15.281Z: Finished operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 07, 2021 7:01:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:15.393Z: Executing operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Values/Values/Map+Collect write end time+Match all files/Match filepatterns+Match all files/Reshuffle.ViaRandomKey/Pair with random key+Match all files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 07, 2021 7:01:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:19.593Z: Finished operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Values/Values/Map+Collect write end time+Match all files/Match filepatterns+Match all files/Reshuffle.ViaRandomKey/Pair with random key+Match all files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 07, 2021 7:01:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:19.683Z: Executing operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 07, 2021 7:01:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:19.730Z: Finished operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
May 07, 2021 7:01:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:19.780Z: Executing operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Create
May 07, 2021 7:01:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:19.959Z: Finished operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Create
May 07, 2021 7:01:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:20.092Z: Executing operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Match all files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Match all files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matches/ParDo(ToReadableFile)+Read files/Read all via FileBasedSource/Split into ranges+Read files/Read all via FileBasedSource/Reshuffle/Pair with random key+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Write
May 07, 2021 7:01:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:21.692Z: Finished operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Match all files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Match all files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matches/ParDo(ToReadableFile)+Read files/Read all via FileBasedSource/Split into ranges+Read files/Read all via FileBasedSource/Reshuffle/Pair with random key+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Write
May 07, 2021 7:01:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:21.751Z: Executing operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Close
May 07, 2021 7:01:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:21.825Z: Finished operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Close
May 07, 2021 7:01:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:21.901Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
May 07, 2021 7:01:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:22.098Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
May 07, 2021 7:01:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:22.250Z: Executing operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Read+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/ExpandIterable+Read files/Read all via FileBasedSource/Reshuffle/Values/Values/Map+Read files/Read all via FileBasedSource/Read ranges+Collect read end time+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
May 07, 2021 7:01:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:40.883Z: Finished operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Read+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/ExpandIterable+Read files/Read all via FileBasedSource/Reshuffle/Values/Values/Map+Read files/Read all via FileBasedSource/Read ranges+Collect read end time+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
May 07, 2021 7:01:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:40.950Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Close
May 07, 2021 7:01:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:41.040Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Close
May 07, 2021 7:01:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:41.149Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
May 07, 2021 7:01:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:41.259Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Create
May 07, 2021 7:01:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:42.141Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Create
May 07, 2021 7:01:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:42.183Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
May 07, 2021 7:01:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:42.481Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate hashcode/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+Calculate hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)
May 07, 2021 7:01:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:42.511Z: Executing operation PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 07, 2021 7:01:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:42.790Z: Finished operation PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
May 07, 2021 7:01:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:44.710Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate hashcode/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+Calculate hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)
May 07, 2021 7:01:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:44.925Z: Executing operation Calculate hashcode/View.AsIterable/CreateDataflowView
May 07, 2021 7:01:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:44.968Z: Finished operation Calculate hashcode/View.AsIterable/CreateDataflowView
May 07, 2021 7:01:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:45.105Z: Executing operation Calculate hashcode/CreateVoid/Read(CreateSource)+Calculate hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
May 07, 2021 7:01:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:45.783Z: Finished operation Calculate hashcode/CreateVoid/Read(CreateSource)+Calculate hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
May 07, 2021 7:01:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:45.850Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
May 07, 2021 7:01:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:45.908Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
May 07, 2021 7:01:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:45.936Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Close
May 07, 2021 7:01:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:45.969Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
May 07, 2021 7:01:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:45.980Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Close
May 07, 2021 7:01:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:46.055Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
May 07, 2021 7:01:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:46.169Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
May 07, 2021 7:01:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:46.302Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
May 07, 2021 7:01:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:46.816Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
May 07, 2021 7:01:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:47.813Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
May 07, 2021 7:01:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:48.085Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
May 07, 2021 7:01:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:48.216Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
May 07, 2021 7:01:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:48.282Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
May 07, 2021 7:01:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:50.553Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
May 07, 2021 7:01:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:50.696Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
May 07, 2021 7:01:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:50.741Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
May 07, 2021 7:01:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:50.871Z: Executing operation Delete test files
May 07, 2021 7:01:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:53.888Z: Finished operation Delete test files
May 07, 2021 7:01:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:54.053Z: Cleaning up.
May 07, 2021 7:01:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:01:54.173Z: Stopping **** pool...
May 07, 2021 7:02:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:02:39.070Z: Autoscaling: Resized **** pool from 5 to 0.
May 07, 2021 7:02:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-07T07:02:39.245Z: Worker pool stopped.
May 07, 2021 7:02:47 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-05-06_23_57_50-1165277662034371950 finished with status DONE.
May 07, 2021 7:02:47 AM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric num_copies, from namespace org.apache.beam.sdk.extensions.gcp.storage.GcsFileSystem
May 07, 2021 7:02:47 AM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric
SEVERE: Failed to get metric copy_time_msec, from namespace org.apache.beam.sdk.extensions.gcp.storage.GcsFileSystem
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED
org.apache.beam.sdk.io.text.TextIOIT > writeThenReadAll FAILED
com.google.cloud.bigquery.BigQueryException: An internal error occurred and the request could not be completed.
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:115)
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.insertAll(HttpBigQueryRpc.java:494)
at com.google.cloud.bigquery.BigQueryImpl.insertAll(BigQueryImpl.java:1036)
at org.apache.beam.sdk.testutils.publishing.BigQueryClient.insertAll(BigQueryClient.java:123)
at org.apache.beam.sdk.testutils.publishing.BigQueryClient.insertAll(BigQueryClient.java:110)
at org.apache.beam.sdk.testutils.publishing.BigQueryResultsPublisher.publish(BigQueryResultsPublisher.java:57)
at org.apache.beam.sdk.testutils.metrics.IOITMetrics.publish(IOITMetrics.java:69)
at org.apache.beam.sdk.testutils.metrics.IOITMetrics.publish(IOITMetrics.java:59)
at org.apache.beam.sdk.io.text.TextIOIT.collectAndPublishMetrics(TextIOIT.java:174)
at org.apache.beam.sdk.io.text.TextIOIT.writeThenReadAll(TextIOIT.java:162)
Caused by:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 500 Internal Server Error
POST https://www.googleapis.com/bigquery/v2/projects/apache-beam-testing/datasets/beam_performance/tables/many_files_textioit_hdfs_results/insertAll?prettyPrint=false
{
"code" : 500,
"errors" : [ {
"domain" : "global",
"message" : "An internal error occurred and the request could not be completed.",
"reason" : "internalError"
} ],
"message" : "An internal error occurred and the request could not be completed.",
"status" : "INTERNAL"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:428)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:514)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:455)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:565)
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.insertAll(HttpBigQueryRpc.java:492)
... 8 more
1 test completed, 1 failed
Finished generating test XML results (0.029 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT_HDFS/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.036 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT_HDFS/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 4,5,main]) completed. Took 5 mins 15.642 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT_HDFS/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 46s
125 actionable tasks: 76 executed, 49 from cache
Publishing build scan...
https://gradle.com/s/xspo5wlnft2ni
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_PerformanceTests_ManyFiles_TextIOIT_HDFS #3422
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT_HDFS/3422/display/redirect>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org