You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/09/26 18:24:47 UTC
Build failed in Jenkins: beam_PerformanceTests_HadoopFormat #3545
See <https://ci-beam.apache.org/job/beam_PerformanceTests_HadoopFormat/3545/display/redirect>
Changes:
------------------------------------------
[...truncated 325.76 KB...]
INFO: Adding Write using Hadoop OutputFormat/CommitWriteJob as step s28
Sep 26, 2021 6:20:35 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.34.0-SNAPSHOT
Sep 26, 2021 6:20:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-09-26_11_20_35-11041036747233291302?project=apache-beam-testing
Sep 26, 2021 6:20:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-09-26_11_20_35-11041036747233291302
Sep 26, 2021 6:20:37 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-09-26_11_20_35-11041036747233291302
Sep 26, 2021 6:20:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-09-26T18:20:40.890Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: hadoopformatioit0writeandreadusinghadoopformat-jenkins-092-k8hf. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Sep 26, 2021 6:20:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:54.975Z: Worker configuration: e2-standard-2 in us-central1-a.
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:55.665Z: Expanding CoGroupByKey operations into optimizable parts.
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:55.868Z: Expanding GroupByKey operations into optimizable parts.
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:55.893Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.096Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.122Z: Fusing consumer Produce db rows into Generate sequence/Read(BoundedCountingSource)
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.155Z: Fusing consumer Prevent fusion before writing/Pair with random key into Produce db rows
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.181Z: Fusing consumer Prevent fusion before writing/Reshuffle/Window.Into()/Window.Assign into Prevent fusion before writing/Pair with random key
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.214Z: Fusing consumer Prevent fusion before writing/Reshuffle/GroupByKey/Reify into Prevent fusion before writing/Reshuffle/Window.Into()/Window.Assign
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.245Z: Fusing consumer Prevent fusion before writing/Reshuffle/GroupByKey/Write into Prevent fusion before writing/Reshuffle/GroupByKey/Reify
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.272Z: Fusing consumer Prevent fusion before writing/Reshuffle/GroupByKey/GroupByWindow into Prevent fusion before writing/Reshuffle/GroupByKey/Read
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.302Z: Fusing consumer Prevent fusion before writing/Reshuffle/ExpandIterable into Prevent fusion before writing/Reshuffle/GroupByKey/GroupByWindow
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.330Z: Fusing consumer Prevent fusion before writing/Values/Values/Map into Prevent fusion before writing/Reshuffle/ExpandIterable
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.360Z: Fusing consumer Collect write time into Prevent fusion before writing/Values/Values/Map
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.392Z: Fusing consumer Construct rows for DBOutputFormat into Collect write time
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.426Z: Fusing consumer Write using Hadoop OutputFormat/ParDo(SetupJob) into Construct rows for DBOutputFormat
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.473Z: Fusing consumer Write using Hadoop OutputFormat/GroupDataByPartition/AssignTask into Write using Hadoop OutputFormat/ParDo(SetupJob)
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.522Z: Fusing consumer Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/Reify into Write using Hadoop OutputFormat/GroupDataByPartition/AssignTask
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.569Z: Fusing consumer Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/Write into Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/Reify
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.599Z: Fusing consumer Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/GroupByWindow into Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/Read
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.633Z: Fusing consumer Write using Hadoop OutputFormat/GroupDataByPartition/FlattenGroupedTasks into Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/GroupByWindow
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.674Z: Fusing consumer Write using Hadoop OutputFormat/Write into Write using Hadoop OutputFormat/GroupDataByPartition/FlattenGroupedTasks
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.706Z: Fusing consumer Write using Hadoop OutputFormat/CollectWriteTasks/WithKeys/AddKeys/Map into Write using Hadoop OutputFormat/Write
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.733Z: Fusing consumer Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey+Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/Combine.GroupedValues/Partial into Write using Hadoop OutputFormat/CollectWriteTasks/WithKeys/AddKeys/Map
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.755Z: Fusing consumer Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey/Reify into Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey+Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/Combine.GroupedValues/Partial
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.790Z: Fusing consumer Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey/Write into Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey/Reify
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.813Z: Fusing consumer Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/Combine.GroupedValues into Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey/Read
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.843Z: Fusing consumer Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/Combine.GroupedValues/Extract into Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/Combine.GroupedValues
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.878Z: Fusing consumer Write using Hadoop OutputFormat/CollectWriteTasks/Values/Values/Map into Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/Combine.GroupedValues/Extract
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.901Z: Fusing consumer Write using Hadoop OutputFormat/CommitWriteJob into Write using Hadoop OutputFormat/CollectWriteTasks/Values/Values/Map
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.934Z: Fusing consumer Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map into Write using Hadoop OutputFormat/CreateOutputConfig/Read(CreateSource)
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.960Z: Fusing consumer Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial into Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:56.993Z: Fusing consumer Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify into Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:57.028Z: Fusing consumer Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write into Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:57.084Z: Fusing consumer Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues into Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:57.111Z: Fusing consumer Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract into Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:57.143Z: Fusing consumer Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map into Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:57.173Z: Fusing consumer Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:57.199Z: Fusing consumer Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:57.232Z: Fusing consumer Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:57.589Z: Executing operation Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:57.620Z: Executing operation Prevent fusion before writing/Reshuffle/GroupByKey/Create
Sep 26, 2021 6:20:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:57.673Z: Starting 5 ****s in us-central1-a...
Sep 26, 2021 6:21:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:58.152Z: Finished operation Prevent fusion before writing/Reshuffle/GroupByKey/Create
Sep 26, 2021 6:21:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:58.152Z: Finished operation Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Sep 26, 2021 6:21:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:58.342Z: Executing operation Write using Hadoop OutputFormat/CreateOutputConfig/Read(CreateSource)+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
Sep 26, 2021 6:21:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:20:58.378Z: Executing operation Generate sequence/Read(BoundedCountingSource)+Produce db rows+Prevent fusion before writing/Pair with random key+Prevent fusion before writing/Reshuffle/Window.Into()/Window.Assign+Prevent fusion before writing/Reshuffle/GroupByKey/Reify+Prevent fusion before writing/Reshuffle/GroupByKey/Write
Sep 26, 2021 6:21:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:21:11.128Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Sep 26, 2021 6:21:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:21:27.981Z: Autoscaling: Raised the number of ****s to 4 based on the rate of progress in the currently running stage(s).
Sep 26, 2021 6:21:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:21:28.007Z: Resized **** pool to 4, though goal was 5. This could be a quota issue.
Sep 26, 2021 6:21:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:21:38.381Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
Sep 26, 2021 6:22:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:01.862Z: Workers have started successfully.
Sep 26, 2021 6:22:03 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:01.897Z: Workers have started successfully.
Sep 26, 2021 6:22:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:29.926Z: Finished operation Write using Hadoop OutputFormat/CreateOutputConfig/Read(CreateSource)+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
Sep 26, 2021 6:22:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:29.982Z: Executing operation Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Sep 26, 2021 6:22:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:30.034Z: Finished operation Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
Sep 26, 2021 6:22:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:30.102Z: Executing operation Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Sep 26, 2021 6:22:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:30.246Z: Finished operation Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Sep 26, 2021 6:22:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:30.383Z: Executing operation Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Sep 26, 2021 6:22:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:32.420Z: Finished operation Generate sequence/Read(BoundedCountingSource)+Produce db rows+Prevent fusion before writing/Pair with random key+Prevent fusion before writing/Reshuffle/Window.Into()/Window.Assign+Prevent fusion before writing/Reshuffle/GroupByKey/Reify+Prevent fusion before writing/Reshuffle/GroupByKey/Write
Sep 26, 2021 6:22:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:32.488Z: Executing operation Prevent fusion before writing/Reshuffle/GroupByKey/Close
Sep 26, 2021 6:22:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:32.544Z: Finished operation Prevent fusion before writing/Reshuffle/GroupByKey/Close
Sep 26, 2021 6:22:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:34.398Z: Finished operation Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
Sep 26, 2021 6:22:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:34.460Z: Executing operation Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Sep 26, 2021 6:22:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:34.506Z: Finished operation Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
Sep 26, 2021 6:22:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:34.574Z: Executing operation Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Sep 26, 2021 6:22:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:36.922Z: Finished operation Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
Sep 26, 2021 6:22:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:37.046Z: Executing operation Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
Sep 26, 2021 6:22:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:37.098Z: Finished operation Write using Hadoop OutputFormat/View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
Sep 26, 2021 6:22:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:37.215Z: Executing operation Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/Create
Sep 26, 2021 6:22:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:37.350Z: Finished operation Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/Create
Sep 26, 2021 6:22:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:37.470Z: Executing operation Prevent fusion before writing/Reshuffle/GroupByKey/Read+Prevent fusion before writing/Reshuffle/GroupByKey/GroupByWindow+Prevent fusion before writing/Reshuffle/ExpandIterable+Prevent fusion before writing/Values/Values/Map+Collect write time+Construct rows for DBOutputFormat+Write using Hadoop OutputFormat/ParDo(SetupJob)+Write using Hadoop OutputFormat/GroupDataByPartition/AssignTask+Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/Reify+Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/Write
Sep 26, 2021 6:22:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:47.049Z: Finished operation Prevent fusion before writing/Reshuffle/GroupByKey/Read+Prevent fusion before writing/Reshuffle/GroupByKey/GroupByWindow+Prevent fusion before writing/Reshuffle/ExpandIterable+Prevent fusion before writing/Values/Values/Map+Collect write time+Construct rows for DBOutputFormat+Write using Hadoop OutputFormat/ParDo(SetupJob)+Write using Hadoop OutputFormat/GroupDataByPartition/AssignTask+Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/Reify+Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/Write
Sep 26, 2021 6:22:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:47.113Z: Executing operation Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/Close
Sep 26, 2021 6:22:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:47.447Z: Finished operation Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/Close
Sep 26, 2021 6:22:47 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:47.507Z: Executing operation Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey/Create
Sep 26, 2021 6:22:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:47.629Z: Finished operation Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey/Create
Sep 26, 2021 6:22:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:22:47.756Z: Executing operation Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/Read+Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/GroupByWindow+Write using Hadoop OutputFormat/GroupDataByPartition/FlattenGroupedTasks+Write using Hadoop OutputFormat/Write+Write using Hadoop OutputFormat/CollectWriteTasks/WithKeys/AddKeys/Map+Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey+Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/Combine.GroupedValues/Partial+Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey/Reify+Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey/Write
Sep 26, 2021 6:23:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:23:08.214Z: Finished operation Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/Read+Write using Hadoop OutputFormat/GroupDataByPartition/GroupByTaskId/GroupByWindow+Write using Hadoop OutputFormat/GroupDataByPartition/FlattenGroupedTasks+Write using Hadoop OutputFormat/Write+Write using Hadoop OutputFormat/CollectWriteTasks/WithKeys/AddKeys/Map+Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey+Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/Combine.GroupedValues/Partial+Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey/Reify+Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey/Write
Sep 26, 2021 6:23:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:23:08.299Z: Executing operation Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey/Close
Sep 26, 2021 6:23:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:23:08.368Z: Finished operation Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey/Close
Sep 26, 2021 6:23:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:23:08.434Z: Executing operation Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey/Read+Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/Combine.GroupedValues+Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/Combine.GroupedValues/Extract+Write using Hadoop OutputFormat/CollectWriteTasks/Values/Values/Map+Write using Hadoop OutputFormat/CommitWriteJob
Sep 26, 2021 6:23:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:23:09.918Z: Finished operation Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/GroupByKey/Read+Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/Combine.GroupedValues+Write using Hadoop OutputFormat/CollectWriteTasks/Combine.perKey(IterableCombiner)/Combine.GroupedValues/Extract+Write using Hadoop OutputFormat/CollectWriteTasks/Values/Values/Map+Write using Hadoop OutputFormat/CommitWriteJob
Sep 26, 2021 6:23:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:23:10.068Z: Cleaning up.
Sep 26, 2021 6:23:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-09-26T18:23:10.295Z: Stopping **** pool...
org.apache.beam.sdk.io.hadoop.format.HadoopFormatIOIT > writeAndReadUsingHadoopFormat SKIPPED
> Task :sdks:java:io:hadoop-format:integrationTest FAILED
:sdks:java:io:hadoop-format:integrationTest (Thread[Execution **** for ':' Thread 2,5,main]) completed. Took 4 mins 36.828 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:hadoop-format:integrationTest'.
> Process 'Gradle Test Executor 2' finished with non-zero exit value 143
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at https://docs.gradle.org/6.9.1/userguide/java_testing.html#sec:test_execution
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 11s
137 actionable tasks: 85 executed, 52 from cache
Publishing build scan...
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=4f6fae2c-db9f-4f18-acae-63cf170d6d06, currentDir=<https://ci-beam.apache.org/job/beam_PerformanceTests_HadoopFormat/ws/src}>
Attempting to read last messages from the daemon log...
Daemon pid: 1885
log file: /home/jenkins/.gradle/daemon/6.9.1/daemon-1885.out.log
----- Last 20 lines from daemon log file - daemon-1885.out.log -----
* What went wrong:
Execution failed for task ':sdks:java:io:hadoop-format:integrationTest'.
> Process 'Gradle Test Executor 2' finished with non-zero exit value 143
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at https://docs.gradle.org/6.9.1/userguide/java_testing.html#sec:test_execution
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 5m 11s
137 actionable tasks: 85 executed, 52 from cache
Publishing build scan...
Daemon vm is shutting down... The daemon has exited normally or was terminated in response to a user interrupt.
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal :
beam_PerformanceTests_HadoopFormat #3546
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_HadoopFormat/3546/display/redirect>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org