You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2020/07/15 13:04:00 UTC
Build failed in Jenkins:
beam_PerformanceTests_ManyFiles_TextIOIT_HDFS #2237
See <https://ci-beam.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT_HDFS/2237/display/redirect?page=changes>
Changes:
[Kamil Wasilewski] [BEAM-9889] Publish InfluxDB backup to public GCS bucket
[Kamil Wasilewski] [BEAM-9889] Populate local instance of InfluxDB with data
------------------------------------------
[...truncated 327.21 KB...]
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:29.104Z: Fusing consumer Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:29.163Z: Fusing consumer Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:29.213Z: Fusing consumer Match all files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:29.274Z: Fusing consumer Match all files/Reshuffle.ViaRandomKey/Values/Values/Map into Match all files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:29.337Z: Fusing consumer Read matches/ParDo(ToReadableFile) into Match all files/Reshuffle.ViaRandomKey/Values/Values/Map
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:29.400Z: Fusing consumer Read files/Read all via FileBasedSource/Split into ranges into Read matches/ParDo(ToReadableFile)
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:29.458Z: Fusing consumer Read files/Read all via FileBasedSource/Reshuffle/Pair with random key into Read files/Read all via FileBasedSource/Split into ranges
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:29.514Z: Fusing consumer Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign into Read files/Read all via FileBasedSource/Reshuffle/Pair with random key
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:29.578Z: Fusing consumer Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify into Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:29.636Z: Fusing consumer Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Write into Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:29.696Z: Fusing consumer Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow into Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Read
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:29.770Z: Fusing consumer Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/ExpandIterable into Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:29.826Z: Fusing consumer Read files/Read all via FileBasedSource/Reshuffle/Values/Values/Map into Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/ExpandIterable
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:29.885Z: Fusing consumer Read files/Read all via FileBasedSource/Read ranges into Read files/Read all via FileBasedSource/Reshuffle/Values/Values/Map
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:29.954Z: Fusing consumer Collect read end time into Read files/Read all via FileBasedSource/Read ranges
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:30.011Z: Fusing consumer Calculate hashcode/WithKeys/AddKeys/Map into Collect read end time
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:30.070Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate hashcode/WithKeys/AddKeys/Map
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:30.117Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:30.160Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:30.195Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:30.228Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:30.267Z: Fusing consumer Calculate hashcode/Values/Values/Map into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Jul 15, 2020 12:58:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:30.299Z: Fusing consumer Calculate hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:30.335Z: Fusing consumer PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:30.359Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/Reify into PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:30.879Z: Executing operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:30.918Z: Executing operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:30.949Z: Executing operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Create
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:30.962Z: Starting 5 ****s in us-central1-a...
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:30.987Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:31.010Z: Finished operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:31.019Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Create
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:31.035Z: Finished operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:31.035Z: Finished operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Create
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:31.060Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:31.064Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:31.098Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Create
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:31.104Z: Executing operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Create
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:31.133Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:31.145Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:31.176Z: Finished operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Create
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:31.223Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:31.490Z: Executing operation PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
Jul 15, 2020 12:58:32 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T12:58:31.524Z: Executing operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Collect write start time+Write content to files/WriteFiles/RewindowIntoGlobal/Window.Assign+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/ApplyShardingKey+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Reify+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Write
Jul 15, 2020 12:58:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-07-15T12:58:32.939Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Jul 15, 2020 1:00:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:00:58.065Z: Workers have started successfully.
Jul 15, 2020 1:01:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:01:27.496Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
Jul 15, 2020 1:01:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:01:28.308Z: Workers have started successfully.
Jul 15, 2020 1:01:46 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:01:44.182Z: Finished operation PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write
Jul 15, 2020 1:02:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:02:02.583Z: Finished operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Collect write start time+Write content to files/WriteFiles/RewindowIntoGlobal/Window.Assign+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/ApplyShardingKey+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Reify+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Write
Jul 15, 2020 1:02:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:02:02.699Z: Executing operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Close
Jul 15, 2020 1:02:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:02:02.777Z: Finished operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Close
Jul 15, 2020 1:02:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:02:02.857Z: Executing operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Read+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/GroupByWindow+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/WriteShardsIntoTempFiles+Write content to files/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
Jul 15, 2020 1:03:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:06.330Z: Finished operation Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/Read+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/GroupIntoShards/GroupByWindow+Write content to files/WriteFiles/WriteShardedBundlesToTempFiles/WriteShardsIntoTempFiles+Write content to files/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
Jul 15, 2020 1:03:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:06.561Z: Executing operation Write content to files/WriteFiles/GatherTempFileResults/View.AsList/CreateDataflowView
Jul 15, 2020 1:03:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:06.647Z: Finished operation Write content to files/WriteFiles/GatherTempFileResults/View.AsList/CreateDataflowView
Jul 15, 2020 1:03:07 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:06.855Z: Executing operation Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+Write content to files/WriteFiles/FinalizeTempFileBundles/Finalize+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Jul 15, 2020 1:03:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:15.693Z: Finished operation Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+Write content to files/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+Write content to files/WriteFiles/FinalizeTempFileBundles/Finalize+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Jul 15, 2020 1:03:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:15.923Z: Executing operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Jul 15, 2020 1:03:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:16.172Z: Finished operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Jul 15, 2020 1:03:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:16.388Z: Executing operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Values/Values/Map+Collect write end time+Match all files/Match filepatterns+Match all files/Reshuffle.ViaRandomKey/Pair with random key+Match all files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Jul 15, 2020 1:03:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:20.333Z: Finished operation Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write content to files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Values/Values/Map+Collect write end time+Match all files/Match filepatterns+Match all files/Reshuffle.ViaRandomKey/Pair with random key+Match all files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Jul 15, 2020 1:03:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:20.457Z: Executing operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Jul 15, 2020 1:03:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:20.525Z: Finished operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Jul 15, 2020 1:03:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:20.618Z: Executing operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Match all files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Match all files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matches/ParDo(ToReadableFile)+Read files/Read all via FileBasedSource/Split into ranges+Read files/Read all via FileBasedSource/Reshuffle/Pair with random key+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Write
Jul 15, 2020 1:03:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:22.810Z: Finished operation Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Match all files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Match all files/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Match all files/Reshuffle.ViaRandomKey/Values/Values/Map+Read matches/ParDo(ToReadableFile)+Read files/Read all via FileBasedSource/Split into ranges+Read files/Read all via FileBasedSource/Reshuffle/Pair with random key+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Write
Jul 15, 2020 1:03:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:22.907Z: Executing operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Close
Jul 15, 2020 1:03:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:22.969Z: Finished operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Close
Jul 15, 2020 1:03:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:23.050Z: Executing operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Read+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/ExpandIterable+Read files/Read all via FileBasedSource/Reshuffle/Values/Values/Map+Read files/Read all via FileBasedSource/Read ranges+Collect read end time+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Jul 15, 2020 1:03:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:50.185Z: Finished operation Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Read+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+Read files/Read all via FileBasedSource/Reshuffle/Reshuffle/ExpandIterable+Read files/Read all via FileBasedSource/Reshuffle/Values/Values/Map+Read files/Read all via FileBasedSource/Read ranges+Collect read end time+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Jul 15, 2020 1:03:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:50.265Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Close
Jul 15, 2020 1:03:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:50.324Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Close
Jul 15, 2020 1:03:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:50.389Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate hashcode/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+Calculate hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)
Jul 15, 2020 1:03:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:53.669Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate hashcode/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+Calculate hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)
Jul 15, 2020 1:03:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:54.003Z: Executing operation Calculate hashcode/View.AsIterable/CreateDataflowView
Jul 15, 2020 1:03:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:54.071Z: Finished operation Calculate hashcode/View.AsIterable/CreateDataflowView
Jul 15, 2020 1:03:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:54.218Z: Executing operation Calculate hashcode/CreateVoid/Read(CreateSource)+Calculate hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
Jul 15, 2020 1:03:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:54.853Z: Finished operation Calculate hashcode/CreateVoid/Read(CreateSource)+Calculate hashcode/ProduceDefault+PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)+PAssert$0/GroupGlobally/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GroupByKey/Reify+PAssert$0/GroupGlobally/GroupByKey/Write+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
Jul 15, 2020 1:03:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:54.928Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Jul 15, 2020 1:03:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:54.976Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Close
Jul 15, 2020 1:03:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:54.988Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Jul 15, 2020 1:03:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:55.022Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Close
Jul 15, 2020 1:03:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:55.062Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Jul 15, 2020 1:03:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:55.100Z: Executing operation PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
Jul 15, 2020 1:03:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:55.717Z: Finished operation PAssert$0/GroupGlobally/GroupByKey/Read+PAssert$0/GroupGlobally/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
Jul 15, 2020 1:03:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:55.750Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Jul 15, 2020 1:03:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:55.871Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Jul 15, 2020 1:03:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:55.927Z: Finished operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Jul 15, 2020 1:03:56 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-07-15T13:03:56.045Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Jul 15, 2020 1:03:59 PM org.apache.beam.runners.dataflow.DataflowPipelineJob lambda$waitUntilFinish$0
WARNING: Job is already running in Google Cloud Platform, Ctrl-C will not cancel it.
To cancel the job in the cloud, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2020-07-15_05_58_09-1437552358515980780
org.apache.beam.sdk.io.text.TextIOIT > writeThenReadAll SKIPPED
> Task :sdks:java:io:file-based-io-tests:integrationTest FAILED
:sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** for ':' Thread 8,5,main]) completed. Took 6 mins 2.737 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> Process 'Gradle Test Executor 3' finished with non-zero exit value 143
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at https://docs.gradle.org/5.2.1/userguide/java_testing.html#sec:test_execution
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 6m 41s
95 actionable tasks: 60 executed, 35 from cache
Publishing build scan...
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=ce1668af-ebb6-4503-92f2-0e3670426549, currentDir=<https://ci-beam.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT_HDFS/ws/src}>
Attempting to read last messages from the daemon log...
Daemon pid: 21478
log file: /home/jenkins/.gradle/daemon/5.2.1/daemon-21478.out.log
----- Last 20 lines from daemon log file - daemon-21478.out.log -----
* What went wrong:
Execution failed for task ':sdks:java:io:file-based-io-tests:integrationTest'.
> Process 'Gradle Test Executor 3' finished with non-zero exit value 143
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at https://docs.gradle.org/5.2.1/userguide/java_testing.html#sec:test_execution
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 6m 41s
95 actionable tasks: 60 executed, 35 from cache
Publishing build scan...
Daemon vm is shutting down... The daemon has exited normally or was terminated in response to a user interrupt.
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_PerformanceTests_ManyFiles_TextIOIT_HDFS #2238
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT_HDFS/2238/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org