You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/04/19 11:30:08 UTC
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1346
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1346/display/redirect?page=changes>
Changes:
[noreply] [BEAM-8611] Move TextSourceTest into TextIOReadTest (#14560)
------------------------------------------
[...truncated 3.69 MB...]
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:13.866Z: Expanding GroupByKey operations into optimizable parts.
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:13.890Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.010Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.030Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.055Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.084Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.108Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.129Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.149Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.165Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.186Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.210Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.229Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.254Z: Unzipping flatten s12 for input s11.org.apache.beam.sdk.values.PCollection.<init>:402#5f2ef1f005ae0b4
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.302Z: Fusing unzipped copy of WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.328Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.343Z: Fusing consumer ParseGameEvent into TextIO.Read/Read
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.366Z: Fusing consumer ExtractUserScore/MapElements/Map into ParseGameEvent
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.386Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial into ExtractUserScore/MapElements/Map
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.412Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.437Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.456Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.477Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.518Z: Fusing consumer WriteUserScoreSums/ConvertToRow into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.543Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign into WriteUserScoreSums/ConvertToRow
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.589Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.609Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.629Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
Apr 19, 2021 11:27:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.654Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
Apr 19, 2021 11:27:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.676Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
Apr 19, 2021 11:27:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.699Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
Apr 19, 2021 11:27:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:14.996Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 19, 2021 11:27:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:15.057Z: Starting 1 workers in us-central1-f...
Apr 19, 2021 11:27:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:15.418Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 19, 2021 11:27:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:15.558Z: Executing operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 19, 2021 11:27:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:27:39.246Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 19, 2021 11:28:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:28:07.636Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 19, 2021 11:28:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:28:33.178Z: Workers have started successfully.
Apr 19, 2021 11:28:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:28:33.195Z: Workers have started successfully.
Apr 19, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-19T11:28:41.612Z: java.lang.NoClassDefFoundError: org/apache/beam/sdk/metrics/MetricsLogger
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.metrics.MetricsLogger
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 28 more
Apr 19, 2021 11:28:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-19T11:28:47.147Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 19, 2021 11:28:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-19T11:28:53.172Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 19, 2021 11:29:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-19T11:29:01.431Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 19, 2021 11:29:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:29:01.456Z: Finished operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 19, 2021 11:29:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-19T11:29:01.543Z: Workflow failed. Causes: S02:TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write failed., Internal Issue (e1b32daae046c3a5): 63963027:24514
Apr 19, 2021 11:29:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:29:01.596Z: Cleaning up.
Apr 19, 2021 11:29:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:29:01.688Z: Stopping worker pool...
Apr 19, 2021 11:29:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:29:51.268Z: Autoscaling: Resized worker pool from 1 to 0.
Apr 19, 2021 11:29:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-19T11:29:51.293Z: Worker pool stopped.
Apr 19, 2021 11:30:04 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-04-19_04_27_01-13629697441774317891 failed with status FAILED.
gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:direct-java:runMobileGamingJavaDirect'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 29m 46s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/kjn5wm52extgs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostRelease_NightlySnapshot
#1366
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1366/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostRelease_NightlySnapshot - Build # 1365 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostRelease_NightlySnapshot - Build # 1365 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1365/ to view the results.
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1364
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1364/display/redirect?page=changes>
Changes:
[shoyer] Use __getstate__ & __setstate__ for deterministic coding
[shoyer] Re-raise exceptions during deterministic encoding
[shoyer] Add test for non-deterministic inputs
[aromanenko.dev] [BEAM-12271] Make max width for code snippets adaptive with text blocks
[Kenneth Knowles] [BEAM-12273] Support non-multimap materialization in Twister2 runner
[Ismaël Mejía] [BEAM-12280] Upgrade Flink runner to Flink version 1.12.3
[noreply] [BEAM-11759] Create Beam glossary (#14717)
------------------------------------------
[...truncated 3.69 MB...]
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-05-05_04_24_26-5692170606189967423
May 05, 2021 11:24:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:30.269Z: Autoscaling is enabled for job 2021-05-05_04_24_26-5692170606189967423. The number of workers will be between 1 and 1000.
May 05, 2021 11:24:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:30.449Z: Autoscaling was automatically enabled for job 2021-05-05_04_24_26-5692170606189967423.
May 05, 2021 11:24:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:39.622Z: Worker configuration: n1-standard-1 in us-central1-c.
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:40.660Z: Expanding CoGroupByKey operations into optimizable parts.
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:40.822Z: Expanding GroupByKey operations into optimizable parts.
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:40.855Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.116Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.144Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.177Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.204Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.241Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.275Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.301Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.339Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.375Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.398Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.433Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.461Z: Unzipping flatten s12 for input s11.org.apache.beam.sdk.values.PCollection.<init>:402#5f2ef1f005ae0b4
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.500Z: Fusing unzipped copy of WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.530Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.566Z: Fusing consumer ParseGameEvent into TextIO.Read/Read
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.604Z: Fusing consumer ExtractUserScore/MapElements/Map into ParseGameEvent
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.643Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial into ExtractUserScore/MapElements/Map
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.678Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.716Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.749Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.784Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.821Z: Fusing consumer WriteUserScoreSums/ConvertToRow into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.850Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign into WriteUserScoreSums/ConvertToRow
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.893Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.948Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:41.984Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:42.011Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:42.045Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:42.081Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:42.492Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:42.590Z: Starting 1 workers in us-central1-c...
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:42.967Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
May 05, 2021 11:24:43 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:24:43.182Z: Executing operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
May 05, 2021 11:25:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:25:12.698Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 05, 2021 11:25:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:25:13.869Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
May 05, 2021 11:25:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:25:49.987Z: Workers have started successfully.
May 05, 2021 11:25:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:25:50.028Z: Workers have started successfully.
May 05, 2021 11:26:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:09.078Z: Finished operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
May 05, 2021 11:26:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:09.159Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Close
May 05, 2021 11:26:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:09.244Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Close
May 05, 2021 11:26:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:09.309Z: Executing operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
May 05, 2021 11:26:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:09.542Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
May 05, 2021 11:26:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:09.788Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract+WriteUserScoreSums/ConvertToRow+WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
May 05, 2021 11:26:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:19.302Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract+WriteUserScoreSums/ConvertToRow+WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
May 05, 2021 11:26:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:19.421Z: Executing operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
May 05, 2021 11:26:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:19.484Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
May 05, 2021 11:26:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:19.566Z: Executing operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
May 05, 2021 11:26:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:22.177Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
May 05, 2021 11:26:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:22.366Z: Executing operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections/Unzipped-1
May 05, 2021 11:26:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:22.455Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections/Unzipped-1
May 05, 2021 11:26:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:22.625Z: Executing operation WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/CreateDataflowView
May 05, 2021 11:26:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:22.692Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/CreateDataflowView
May 05, 2021 11:26:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:22.873Z: Executing operation WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 05, 2021 11:26:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:23.163Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 05, 2021 11:26:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:23.331Z: Executing operation WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 05, 2021 11:26:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-05T11:26:26.883Z: java.lang.IllegalAccessError: tried to access class org.apache.beam.sdk.values.PCollectionViews$SingletonViewFn2 from class org.apache.beam.runners.dataflow.worker.IsmSideInputReader
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.<clinit>(IsmSideInputReader.java:112)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext.lambda$getSideInputReader$0(BatchModeExecutionContext.java:290)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:167)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
May 05, 2021 11:26:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-05T11:26:33.414Z: java.lang.NoClassDefFoundError: Could not initialize class org.apache.beam.runners.dataflow.worker.IsmSideInputReader
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext.lambda$getSideInputReader$0(BatchModeExecutionContext.java:290)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:167)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
May 05, 2021 11:26:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-05T11:26:43.809Z: java.lang.NoClassDefFoundError: Could not initialize class org.apache.beam.runners.dataflow.worker.IsmSideInputReader
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext.lambda$getSideInputReader$0(BatchModeExecutionContext.java:290)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:167)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
May 05, 2021 11:26:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-05T11:26:54.550Z: java.lang.NoClassDefFoundError: Could not initialize class org.apache.beam.runners.dataflow.worker.IsmSideInputReader
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext.lambda$getSideInputReader$0(BatchModeExecutionContext.java:290)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:167)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
May 05, 2021 11:26:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:54.626Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 05, 2021 11:26:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-05T11:26:54.776Z: Workflow failed. Causes: S11:WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write failed., The job failed because a work item has failed 4 times. Look in previous log entries for the cause of each one of the 4 failures. For more information, see https://cloud.google.com/dataflow/docs/guides/common-errors. The work item was attempted on these workers:
userscore-jenkins-0505112-05050424-otyf-harness-gqnr
Root cause: Work item failed.,
userscore-jenkins-0505112-05050424-otyf-harness-gqnr
Root cause: Work item failed.,
userscore-jenkins-0505112-05050424-otyf-harness-gqnr
Root cause: Work item failed.,
userscore-jenkins-0505112-05050424-otyf-harness-gqnr
Root cause: Work item failed.
May 05, 2021 11:26:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:54.902Z: Cleaning up.
May 05, 2021 11:26:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:26:55.099Z: Stopping worker pool...
May 05, 2021 11:27:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:27:40.296Z: Autoscaling: Resized worker pool from 1 to 0.
May 05, 2021 11:27:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-05T11:27:40.337Z: Worker pool stopped.
May 05, 2021 11:27:46 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-05-05_04_24_26-5692170606189967423 failed with status FAILED.
gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:direct-java:runMobileGamingJavaDirect'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 27m 27s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/e6ew4z5vtq4w2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1363
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1363/display/redirect?page=changes>
Changes:
[daniel.o.programmer] Revert "Resolve beam_fn_api experiment for side inputs, using portable
[samuelw] [BEAM-12254] Change nexmark UnboundedEventSource to always calculate
[Alan Myrvold] [BEAM-10912] Increase Jenkins job logs retention from 14 to 30 days
[Brian Hulette] Allow attaching Singleton reason to NonParallelOperation
[Brian Hulette] Add reason to most Singleton requirements
[Brian Hulette] fillna(axis='index', limit=) is order-sensitive
[Brian Hulette] quantile(axis=columns) is parallelizable
[Brian Hulette] shift(axis=index) is order-sensitive
[samuelw] [BEAM-12253] Change Read.UnboundedSourceAsSDFRestrictionTracker to use
[noreply] Fix lint breakage from Merge pull request #13617 from [BEAM-11289]
[Pablo Estrada] Revert "Merge pull request #13617 from [BEAM-11289] [Python] Integrate
[Pablo Estrada] Revert type: ignore caused by PR 13617
[noreply] [BEAM-12029] Clarify the few remaining ambiguous WontImplementErrors
------------------------------------------
[...truncated 3.71 MB...]
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-05-04_04_24_56-14772058864415611488
May 04, 2021 11:25:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:00.086Z: Autoscaling is enabled for job 2021-05-04_04_24_56-14772058864415611488. The number of workers will be between 1 and 1000.
May 04, 2021 11:25:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:00.225Z: Autoscaling was automatically enabled for job 2021-05-04_04_24_56-14772058864415611488.
May 04, 2021 11:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:07.725Z: Worker configuration: n1-standard-1 in us-central1-f.
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:08.523Z: Expanding CoGroupByKey operations into optimizable parts.
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:08.648Z: Expanding GroupByKey operations into optimizable parts.
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:08.674Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:08.851Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:08.878Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:08.901Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:08.933Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:08.963Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:08.987Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.019Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.044Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.072Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.099Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.130Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.152Z: Unzipping flatten s12 for input s11.org.apache.beam.sdk.values.PCollection.<init>:402#5f2ef1f005ae0b4
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.182Z: Fusing unzipped copy of WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.208Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.236Z: Fusing consumer ParseGameEvent into TextIO.Read/Read
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.268Z: Fusing consumer ExtractUserScore/MapElements/Map into ParseGameEvent
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.300Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial into ExtractUserScore/MapElements/Map
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.335Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.366Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.390Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.425Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.456Z: Fusing consumer WriteUserScoreSums/ConvertToRow into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.478Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign into WriteUserScoreSums/ConvertToRow
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.514Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.547Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.582Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.615Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.647Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
May 04, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:09.680Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
May 04, 2021 11:25:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:10.286Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
May 04, 2021 11:25:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:10.360Z: Starting 1 workers in us-central1-f...
May 04, 2021 11:25:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:10.806Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
May 04, 2021 11:25:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:10.972Z: Executing operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
May 04, 2021 11:25:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:26.703Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 04, 2021 11:25:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:25:56.552Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
May 04, 2021 11:26:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:21.086Z: Workers have started successfully.
May 04, 2021 11:26:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:21.117Z: Workers have started successfully.
May 04, 2021 11:26:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:39.887Z: Finished operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
May 04, 2021 11:26:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:39.927Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Close
May 04, 2021 11:26:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:39.972Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Close
May 04, 2021 11:26:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:40.019Z: Executing operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
May 04, 2021 11:26:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:40.199Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
May 04, 2021 11:26:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:40.294Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract+WriteUserScoreSums/ConvertToRow+WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
May 04, 2021 11:26:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:49.726Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract+WriteUserScoreSums/ConvertToRow+WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
May 04, 2021 11:26:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:49.795Z: Executing operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
May 04, 2021 11:26:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:49.842Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
May 04, 2021 11:26:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:49.890Z: Executing operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
May 04, 2021 11:26:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:52.400Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
May 04, 2021 11:26:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:52.489Z: Executing operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections/Unzipped-1
May 04, 2021 11:26:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:52.583Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections/Unzipped-1
May 04, 2021 11:26:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:52.689Z: Executing operation WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/CreateDataflowView
May 04, 2021 11:26:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:52.730Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/CreateDataflowView
May 04, 2021 11:26:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:52.843Z: Executing operation WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 04, 2021 11:26:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:52.979Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
May 04, 2021 11:26:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:26:53.073Z: Executing operation WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 04, 2021 11:26:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-04T11:26:56.696Z: java.lang.IllegalAccessError: tried to access class org.apache.beam.sdk.values.PCollectionViews$SingletonViewFn2 from class org.apache.beam.runners.dataflow.worker.IsmSideInputReader
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.<clinit>(IsmSideInputReader.java:112)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext.lambda$getSideInputReader$0(BatchModeExecutionContext.java:290)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:167)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
May 04, 2021 11:27:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-04T11:27:04.687Z: java.lang.NoClassDefFoundError: Could not initialize class org.apache.beam.runners.dataflow.worker.IsmSideInputReader
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext.lambda$getSideInputReader$0(BatchModeExecutionContext.java:290)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:167)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
May 04, 2021 11:27:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-04T11:27:17.449Z: java.lang.NoClassDefFoundError: Could not initialize class org.apache.beam.runners.dataflow.worker.IsmSideInputReader
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext.lambda$getSideInputReader$0(BatchModeExecutionContext.java:290)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:167)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
May 04, 2021 11:27:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-04T11:27:35.401Z: java.lang.NoClassDefFoundError: Could not initialize class org.apache.beam.runners.dataflow.worker.IsmSideInputReader
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext.lambda$getSideInputReader$0(BatchModeExecutionContext.java:290)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:167)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
May 04, 2021 11:27:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:27:35.426Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
May 04, 2021 11:27:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-04T11:27:35.507Z: Workflow failed. Causes: S11:WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write failed., The job failed because a work item has failed 4 times. Look in previous log entries for the cause of each one of the 4 failures. For more information, see https://cloud.google.com/dataflow/docs/guides/common-errors. The work item was attempted on these workers:
userscore-jenkins-0504112-05040424-f8r1-harness-gncw
Root cause: Work item failed.,
userscore-jenkins-0504112-05040424-f8r1-harness-gncw
Root cause: Work item failed.,
userscore-jenkins-0504112-05040424-f8r1-harness-gncw
Root cause: Work item failed.,
userscore-jenkins-0504112-05040424-f8r1-harness-gncw
Root cause: Work item failed.
May 04, 2021 11:27:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:27:35.578Z: Cleaning up.
May 04, 2021 11:27:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:27:35.687Z: Stopping worker pool...
May 04, 2021 11:28:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:28:19.764Z: Autoscaling: Resized worker pool from 1 to 0.
May 04, 2021 11:28:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-04T11:28:19.790Z: Worker pool stopped.
May 04, 2021 11:28:35 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-05-04_04_24_56-14772058864415611488 failed with status FAILED.
gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:direct-java:runMobileGamingJavaDirect'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 28m 17s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/r6fdsrquzyiii
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1362
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1362/display/redirect>
Changes:
------------------------------------------
[...truncated 4.01 MB...]
May 03, 2021 11:23:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:23:26.914Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify
May 03, 2021 11:23:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:23:26.950Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read
May 03, 2021 11:23:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:23:26.971Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues
May 03, 2021 11:23:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:23:26.995Z: Fusing consumer WriteUserScoreSums/ConvertToRow into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract
May 03, 2021 11:23:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:23:27.019Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign into WriteUserScoreSums/ConvertToRow
May 03, 2021 11:23:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:23:27.052Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign
May 03, 2021 11:23:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:23:27.107Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
> Task :runners:direct-java:runQuickstartJavaDirect
May 03, 2021 11:23:29 AM org.apache.beam.sdk.io.FileBasedSource getEstimatedSizeBytes
INFO: Filepattern pom.xml matched 1 files with total size 16568
May 03, 2021 11:23:29 AM org.apache.beam.sdk.io.FileBasedSource split
INFO: Splitting filepattern pom.xml into bundles of size 828 took 1 ms and produced 1 files and 20 bundles
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.WriteFiles$WriteShardsIntoTempFilesFn processElement
INFO: Opening writer 6bc03c07-5a35-4932-a8dc-33be01e61526 for window org.apache.beam.sdk.transforms.windowing.GlobalWindow@66c883f7 pane PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0} destination null
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.WriteFiles$WriteShardsIntoTempFilesFn processElement
INFO: Opening writer e25e088d-c5ae-4813-852b-e65b0430ba9c for window org.apache.beam.sdk.transforms.windowing.GlobalWindow@66c883f7 pane PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0} destination null
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.WriteFiles$WriteShardsIntoTempFilesFn processElement
INFO: Opening writer 709fade0-95c9-4c6b-81de-389e37734e3b for window org.apache.beam.sdk.transforms.windowing.GlobalWindow@66c883f7 pane PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0} destination null
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.WriteFiles$WriteShardsIntoTempFilesFn processElement
INFO: Opening writer 9578c5ff-146b-4194-a672-0447845bc0a7 for window org.apache.beam.sdk.transforms.windowing.GlobalWindow@66c883f7 pane PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0} destination null
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.FileBasedSink$Writer close
INFO: Successfully wrote temporary file /tmp/groovy-generated-2128296651866507227-tmpdir/word-count-beam/.temp-beam-4ea97ef0-f27c-43f9-9a82-cf6cd53bf725/709fade0-95c9-4c6b-81de-389e37734e3b
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.FileBasedSink$Writer close
INFO: Successfully wrote temporary file /tmp/groovy-generated-2128296651866507227-tmpdir/word-count-beam/.temp-beam-4ea97ef0-f27c-43f9-9a82-cf6cd53bf725/9578c5ff-146b-4194-a672-0447845bc0a7
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.FileBasedSink$Writer close
INFO: Successfully wrote temporary file /tmp/groovy-generated-2128296651866507227-tmpdir/word-count-beam/.temp-beam-4ea97ef0-f27c-43f9-9a82-cf6cd53bf725/e25e088d-c5ae-4813-852b-e65b0430ba9c
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.FileBasedSink$Writer close
INFO: Successfully wrote temporary file /tmp/groovy-generated-2128296651866507227-tmpdir/word-count-beam/.temp-beam-4ea97ef0-f27c-43f9-9a82-cf6cd53bf725/6bc03c07-5a35-4932-a8dc-33be01e61526
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.WriteFiles$FinalizeTempFileBundles$FinalizeFn process
INFO: Finalizing 4 file results
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation createMissingEmptyShards
INFO: Finalizing for destination null num shards 4.
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation moveToOutputFiles
INFO: Will copy temporary file FileResult{tempFilename=/tmp/groovy-generated-2128296651866507227-tmpdir/word-count-beam/.temp-beam-4ea97ef0-f27c-43f9-9a82-cf6cd53bf725/6bc03c07-5a35-4932-a8dc-33be01e61526, shard=3, window=org.apache.beam.sdk.transforms.windowing.GlobalWindow@66c883f7, paneInfo=PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0}} to final location /tmp/groovy-generated-2128296651866507227-tmpdir/word-count-beam/counts-00003-of-00004
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation moveToOutputFiles
INFO: Will copy temporary file FileResult{tempFilename=/tmp/groovy-generated-2128296651866507227-tmpdir/word-count-beam/.temp-beam-4ea97ef0-f27c-43f9-9a82-cf6cd53bf725/e25e088d-c5ae-4813-852b-e65b0430ba9c, shard=1, window=org.apache.beam.sdk.transforms.windowing.GlobalWindow@66c883f7, paneInfo=PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0}} to final location /tmp/groovy-generated-2128296651866507227-tmpdir/word-count-beam/counts-00001-of-00004
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation moveToOutputFiles
INFO: Will copy temporary file FileResult{tempFilename=/tmp/groovy-generated-2128296651866507227-tmpdir/word-count-beam/.temp-beam-4ea97ef0-f27c-43f9-9a82-cf6cd53bf725/9578c5ff-146b-4194-a672-0447845bc0a7, shard=0, window=org.apache.beam.sdk.transforms.windowing.GlobalWindow@66c883f7, paneInfo=PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0}} to final location /tmp/groovy-generated-2128296651866507227-tmpdir/word-count-beam/counts-00000-of-00004
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation moveToOutputFiles
INFO: Will copy temporary file FileResult{tempFilename=/tmp/groovy-generated-2128296651866507227-tmpdir/word-count-beam/.temp-beam-4ea97ef0-f27c-43f9-9a82-cf6cd53bf725/709fade0-95c9-4c6b-81de-389e37734e3b, shard=2, window=org.apache.beam.sdk.transforms.windowing.GlobalWindow@66c883f7, paneInfo=PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0}} to final location /tmp/groovy-generated-2128296651866507227-tmpdir/word-count-beam/counts-00002-of-00004
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation removeTemporaryFiles
INFO: Will remove known temporary file /tmp/groovy-generated-2128296651866507227-tmpdir/word-count-beam/.temp-beam-4ea97ef0-f27c-43f9-9a82-cf6cd53bf725/e25e088d-c5ae-4813-852b-e65b0430ba9c
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation removeTemporaryFiles
INFO: Will remove known temporary file /tmp/groovy-generated-2128296651866507227-tmpdir/word-count-beam/.temp-beam-4ea97ef0-f27c-43f9-9a82-cf6cd53bf725/709fade0-95c9-4c6b-81de-389e37734e3b
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation removeTemporaryFiles
INFO: Will remove known temporary file /tmp/groovy-generated-2128296651866507227-tmpdir/word-count-beam/.temp-beam-4ea97ef0-f27c-43f9-9a82-cf6cd53bf725/6bc03c07-5a35-4932-a8dc-33be01e61526
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation removeTemporaryFiles
INFO: Will remove known temporary file /tmp/groovy-generated-2128296651866507227-tmpdir/word-count-beam/.temp-beam-4ea97ef0-f27c-43f9-9a82-cf6cd53bf725/9578c5ff-146b-4194-a672-0447845bc0a7
May 03, 2021 11:23:34 AM org.apache.beam.sdk.io.FileBasedSink$WriteOperation removeTemporaryFiles
WARNING: Failed to match temporary files under: [/tmp/groovy-generated-2128296651866507227-tmpdir/word-count-beam/.temp-beam-4ea97ef0-f27c-43f9-9a82-cf6cd53bf725/].
grep Foundation counts*
counts-00001-of-00004:Foundation: 1
Verified Foundation: 1
[SUCCESS]
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow
May 03, 2021 11:23:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:23:27.135Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
May 03, 2021 11:23:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:23:27.176Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
May 03, 2021 11:23:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:23:27.221Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
May 03, 2021 11:23:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:23:27.276Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
May 03, 2021 11:23:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:23:27.705Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
May 03, 2021 11:23:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:23:27.777Z: Starting 1 workers in us-central1-f...
May 03, 2021 11:23:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:23:28.095Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
May 03, 2021 11:23:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:23:28.253Z: Executing operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
May 03, 2021 11:23:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:23:32.841Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 03, 2021 11:24:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:24:18.268Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
May 03, 2021 11:24:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:24:57.168Z: Workers have started successfully.
May 03, 2021 11:24:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:24:57.205Z: Workers have started successfully.
May 03, 2021 11:25:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-03T11:25:08.022Z: java.lang.NoClassDefFoundError: org/apache/beam/sdk/metrics/MetricsLogger
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.metrics.MetricsLogger
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 28 more
May 03, 2021 11:25:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-03T11:25:13.660Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
May 03, 2021 11:25:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-03T11:25:22.348Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
May 03, 2021 11:25:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-03T11:25:32.008Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
May 03, 2021 11:25:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:25:32.033Z: Finished operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
May 03, 2021 11:25:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-03T11:25:32.159Z: Workflow failed. Causes: S02:TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write failed., Internal Issue (387df2ddad001920): 63963027:24514
May 03, 2021 11:25:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:25:32.228Z: Cleaning up.
May 03, 2021 11:25:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:25:32.310Z: Stopping worker pool...
May 03, 2021 11:26:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:26:11.945Z: Autoscaling: Resized worker pool from 1 to 0.
May 03, 2021 11:26:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-03T11:26:11.988Z: Worker pool stopped.
May 03, 2021 11:26:24 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-05-03_04_23_13-9692232114645001839 failed with status FAILED.
gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 26m 7s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/ccf5amhwmwvua
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1361
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1361/display/redirect>
Changes:
------------------------------------------
[...truncated 4.45 MB...]
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-05-02_04_23_59-18215381666970982577
May 02, 2021 11:24:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:02.425Z: Autoscaling is enabled for job 2021-05-02_04_23_59-18215381666970982577. The number of workers will be between 1 and 1000.
May 02, 2021 11:24:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:02.558Z: Autoscaling was automatically enabled for job 2021-05-02_04_23_59-18215381666970982577.
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:10.208Z: Worker configuration: n1-standard-1 in us-central1-f.
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:10.979Z: Expanding CoGroupByKey operations into optimizable parts.
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.117Z: Expanding GroupByKey operations into optimizable parts.
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.149Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.332Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.364Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.389Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.414Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.447Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.472Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.497Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.523Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.548Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.590Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.618Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.661Z: Unzipping flatten s12 for input s11.org.apache.beam.sdk.values.PCollection.<init>:402#5f2ef1f005ae0b4
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.697Z: Fusing unzipped copy of WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.727Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.751Z: Fusing consumer ParseGameEvent into TextIO.Read/Read
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.773Z: Fusing consumer ExtractUserScore/MapElements/Map into ParseGameEvent
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.799Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial into ExtractUserScore/MapElements/Map
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.825Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.861Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.892Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.915Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.948Z: Fusing consumer WriteUserScoreSums/ConvertToRow into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:11.982Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign into WriteUserScoreSums/ConvertToRow
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:12.015Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:12.114Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
May 02, 2021 11:24:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:12.147Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
May 02, 2021 11:24:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:12.168Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
May 02, 2021 11:24:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:12.189Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
May 02, 2021 11:24:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:12.223Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
May 02, 2021 11:24:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:12.615Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
May 02, 2021 11:24:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:12.714Z: Starting 1 workers in us-central1-f...
May 02, 2021 11:24:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:13.222Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
May 02, 2021 11:24:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:13.369Z: Executing operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
May 02, 2021 11:24:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:24:29.309Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 02, 2021 11:25:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:25:04.747Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
May 02, 2021 11:25:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:25:46.541Z: Workers have started successfully.
May 02, 2021 11:25:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:25:46.571Z: Workers have started successfully.
May 02, 2021 11:25:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-02T11:25:57.800Z: java.lang.NoClassDefFoundError: org/apache/beam/sdk/metrics/MetricsLogger
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.metrics.MetricsLogger
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 28 more
May 02, 2021 11:26:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-02T11:26:06.117Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
May 02, 2021 11:26:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-02T11:26:19.404Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
May 02, 2021 11:26:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-02T11:26:37.128Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
May 02, 2021 11:26:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:26:37.155Z: Finished operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
May 02, 2021 11:26:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-02T11:26:37.245Z: Workflow failed. Causes: S02:TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write failed., Internal Issue (ccf53a5c0f0dacf6): 63963027:24514
May 02, 2021 11:26:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:26:37.358Z: Cleaning up.
May 02, 2021 11:26:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:26:37.485Z: Stopping worker pool...
May 02, 2021 11:27:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:27:25.167Z: Autoscaling: Resized worker pool from 1 to 0.
May 02, 2021 11:27:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-02T11:27:25.223Z: Worker pool stopped.
May 02, 2021 11:27:39 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-05-02_04_23_59-18215381666970982577 failed with status FAILED.
gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 27m 22s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/q5n3clw7yl2q6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1360
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1360/display/redirect?page=changes>
Changes:
[chamikaramj] Updates transform overriding to support parent/child external transforms
[Kyle Weaver] [BEAM-12257] Infer accumulator coder from underlying AggregateFn in
[randomstep] [BEAM-12252] Bump kotlin version to 1.4.32 and improve Kotlin code
[noreply] [BEAM-12165] Add support for ParquetIO Sink to specify an Avro model
[noreply] [BEAM-12018] Initial implementation for melt (#14689)
[noreply] Merge pull request #13617 from [BEAM-11289] [Python] Integrate Google
------------------------------------------
[...truncated 4.52 MB...]
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-05-01_04_24_09-11737044199229175335
May 01, 2021 11:24:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:12.927Z: Autoscaling is enabled for job 2021-05-01_04_24_09-11737044199229175335. The number of workers will be between 1 and 1000.
May 01, 2021 11:24:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:13.026Z: Autoscaling was automatically enabled for job 2021-05-01_04_24_09-11737044199229175335.
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:20.014Z: Worker configuration: n1-standard-1 in us-central1-c.
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:20.684Z: Expanding CoGroupByKey operations into optimizable parts.
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:20.804Z: Expanding GroupByKey operations into optimizable parts.
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:20.837Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.004Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.034Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.058Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.087Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.121Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.155Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.181Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.212Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.249Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.278Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.328Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.356Z: Unzipping flatten s12 for input s11.org.apache.beam.sdk.values.PCollection.<init>:402#5f2ef1f005ae0b4
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.388Z: Fusing unzipped copy of WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.424Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.447Z: Fusing consumer ParseGameEvent into TextIO.Read/Read
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.478Z: Fusing consumer ExtractUserScore/MapElements/Map into ParseGameEvent
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.510Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial into ExtractUserScore/MapElements/Map
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.532Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.558Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.587Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.618Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.703Z: Fusing consumer WriteUserScoreSums/ConvertToRow into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.728Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign into WriteUserScoreSums/ConvertToRow
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.750Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.772Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.804Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.829Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.857Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:21.889Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:22.361Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
May 01, 2021 11:24:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:22.439Z: Starting 1 workers in us-central1-c...
May 01, 2021 11:24:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:22.821Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
May 01, 2021 11:24:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:22.960Z: Executing operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
May 01, 2021 11:24:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:24:47.210Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
May 01, 2021 11:25:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:25:10.906Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
May 01, 2021 11:25:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:25:49.221Z: Workers have started successfully.
May 01, 2021 11:25:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:25:49.241Z: Workers have started successfully.
May 01, 2021 11:25:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-01T11:25:57.654Z: java.lang.NoClassDefFoundError: org/apache/beam/sdk/metrics/MetricsLogger
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.metrics.MetricsLogger
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 28 more
May 01, 2021 11:26:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-01T11:26:03.521Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
May 01, 2021 11:26:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-01T11:26:14.716Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
May 01, 2021 11:26:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-01T11:26:31.763Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
May 01, 2021 11:26:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:26:31.789Z: Finished operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
May 01, 2021 11:26:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-05-01T11:26:31.876Z: Workflow failed. Causes: S02:TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write failed., Internal Issue (df4d5382bba5853e): 63963027:24514
May 01, 2021 11:26:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:26:31.938Z: Cleaning up.
May 01, 2021 11:26:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:26:32.027Z: Stopping worker pool...
May 01, 2021 11:27:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:27:22.019Z: Autoscaling: Resized worker pool from 1 to 0.
May 01, 2021 11:27:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-05-01T11:27:22.049Z: Worker pool stopped.
May 01, 2021 11:27:27 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-05-01_04_24_09-11737044199229175335 failed with status FAILED.
gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 27m 9s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/fja52z7mvucdg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1359
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1359/display/redirect?page=changes>
Changes:
[Kenneth Knowles] Fix kenn at apache email address in blog author list
[randomstep] [BEAM-8706] Bump antlr4 to 4.9.2
[kwu] [BEAM-12227] Update ExternalWorkerServiceTest
[heejong] adding 2.31.0 section to CHANGES.md
[heejong] bugfix for cut_release_branch.sh
[noreply] [BEAM-12238] Make StateBackedIterable Serializable. (#14660)
[noreply] [BEAM-9615] Use UUIDs in Go SDK Schemas (#14665)
[noreply] [BEAM-12174] Propertly close ExecutableStageContext in DoFnOp#close
[noreply] [BEAM-12118] Fix race introduced in QueeuingBeamFnDataClient triggering
[noreply] [BEAM-9547] WontImplement a few more DeferredGroupBy operations (#14675)
[Kenneth Knowles] Add Beam 2.29.0 release blog post
[Kenneth Knowles] Update website for 2.29.0 release
[Kenneth Knowles] [BEAM-12222] Update Dataflow container for side input changes
[Kenneth Knowles] Increase timeout of Dataflow V1 streaming ValidatesRunner
[noreply] [BEAM-12158] Regen go protos & rename internal proto package (#14612)
[noreply] Update golock instructions (#14678)
[daniel.o.programmer] [BEAM-12147][BEAM-9919] Add Go SDK x-lang prefix transform
[noreply] [BEAM-12223] Fix javadoc bug in JdbcIO (#14642)
------------------------------------------
[...truncated 3.65 MB...]
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:22.620Z: Expanding GroupByKey operations into optimizable parts.
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:22.651Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:22.831Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:22.865Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:22.897Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:22.932Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:22.961Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.005Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.037Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.072Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.117Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.153Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.217Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.295Z: Unzipping flatten s12 for input s11.org.apache.beam.sdk.values.PCollection.<init>:402#5f2ef1f005ae0b4
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.346Z: Fusing unzipped copy of WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.402Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.431Z: Fusing consumer ParseGameEvent into TextIO.Read/Read
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.470Z: Fusing consumer ExtractUserScore/MapElements/Map into ParseGameEvent
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.511Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial into ExtractUserScore/MapElements/Map
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.555Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.629Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify
Apr 30, 2021 11:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.680Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read
Apr 30, 2021 11:23:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.743Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues
Apr 30, 2021 11:23:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.778Z: Fusing consumer WriteUserScoreSums/ConvertToRow into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract
Apr 30, 2021 11:23:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.803Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign into WriteUserScoreSums/ConvertToRow
Apr 30, 2021 11:23:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.824Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign
Apr 30, 2021 11:23:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.858Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 30, 2021 11:23:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.884Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
Apr 30, 2021 11:23:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.920Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
Apr 30, 2021 11:23:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:23.957Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
Apr 30, 2021 11:23:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:24.009Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
Apr 30, 2021 11:23:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:24.588Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 30, 2021 11:23:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:24.670Z: Starting 1 workers in us-central1-c...
Apr 30, 2021 11:23:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:25.082Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 30, 2021 11:23:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:25.227Z: Executing operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 30, 2021 11:23:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:23:30.008Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 30, 2021 11:24:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:24:09.361Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 30, 2021 11:24:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:24:39.778Z: Workers have started successfully.
Apr 30, 2021 11:24:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:24:39.822Z: Workers have started successfully.
Apr 30, 2021 11:24:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-30T11:24:49.331Z: java.lang.NoClassDefFoundError: org/apache/beam/sdk/metrics/MetricsLogger
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.metrics.MetricsLogger
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 28 more
Apr 30, 2021 11:24:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-30T11:24:54.922Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 30, 2021 11:25:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-30T11:25:04.098Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 30, 2021 11:25:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-30T11:25:17.413Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 30, 2021 11:25:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:25:17.443Z: Finished operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 30, 2021 11:25:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-30T11:25:17.553Z: Workflow failed. Causes: S02:TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write failed., Internal Issue (a296a7ff17de01aa): 63963027:24514
Apr 30, 2021 11:25:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:25:17.643Z: Cleaning up.
Apr 30, 2021 11:25:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:25:17.810Z: Stopping worker pool...
Apr 30, 2021 11:26:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:26:01.694Z: Autoscaling: Resized worker pool from 1 to 0.
Apr 30, 2021 11:26:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-30T11:26:01.804Z: Worker pool stopped.
Apr 30, 2021 11:26:07 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-04-30_04_23_11-10520366476382284422 failed with status FAILED.
gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:direct-java:runMobileGamingJavaDirect'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 25m 49s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/y456dxkedx5ao
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1358
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1358/display/redirect?page=changes>
Changes:
[heejong] [BEAM-12140] Update artifact information when staging artifacts to gcs
[Kenneth Knowles] Update run_rc_validation.sh to pull RC tag
[yoshiki.obata] [BEAM-7372] cleanup following codes from apache_beam/runners/dataflow
[randomstep] [BEAM-12188] Bump snakeyaml to 1.28
[Kenneth Knowles] Remove WindowFn.getOutputTime
[Brian Hulette] Use try-with-resources in ZipFiles
[samuelw] [BEAM-12229] StreamingModeExecutionContext invalidated
[kwu] [BEAM-12227] Implement ExternalWorkerService#stopWorker to support
[kwu] Add ExternalWorkerServiceTest
[Robert Bradshaw] Refinements to the portability page, reflecting its state of
[aromanenko.dev] [BEAM-12243] TPC-DS: use SQL "substring()" instead of "substr()"
[Brian Hulette] Allow overwriting zip file in packageDirectoriesToStage
[noreply] Merge pull request #14615 from [BEAM-12206] Update BigQuery IO
[chamikaramj] Revert "[BEAM-11994] Refactor BigQueryTornadoes to make more options
[daniel.o.programmer] Revert "[BEAM-3713] Moving integration tests from nose to pytest
[heejong] use get_scheme
[heejong] fix typo, add optional sha256 to url payload
[heejong] fix formatting
[noreply] Add scikit-learn to additional user packages (#14664)
[noreply] Merge pull request #14639: [BEAM-10277][BEAM-12198] Populate encoding
[Ismaël Mejía] [BEAM-12247] Reduce memory/string creations in InMemoryTimerInternals
[Ismaël Mejía] [BEAM-12248] Reduce ArrayList allocation in Row/RowUtils
[heejong] Moving to 2.31.0-SNAPSHOT on master branch.
------------------------------------------
[...truncated 4.41 MB...]
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
Caused by: java.lang.IllegalArgumentException: Unknown type of view requested: SimplePCollectionView{tag=Tag<org.apache.beam.sdk.values.PCollectionViews$SimplePCollectionView.<init>:1261#1abfe6cb40bcf2e7>, viewFn=org.apache.beam.sdk.values.PCollectionViews$ListViewFn2@6e6b3f4d, coder=KvCoder(BigEndianLongCoder,PCollectionViews$ValueOrMetadataCoder(FileBasedSink$FileResultCoder(NullableCoder(GlobalWindow$Coder)),OffsetRange$Coder)), windowMappingFn=GlobalWindowMappingFn{}, pCollection=null}
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.lambda$get$3(IsmSideInputReader.java:357)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4876)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4871)
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.get(IsmSideInputReader.java:336)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:232)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188)
at org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:339)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:212)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:163)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:92)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2021 11:27:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-29T11:27:48.470Z: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Unknown type of view requested: SimplePCollectionView{tag=Tag<org.apache.beam.sdk.values.PCollectionViews$SimplePCollectionView.<init>:1261#1abfe6cb40bcf2e7>, viewFn=org.apache.beam.sdk.values.PCollectionViews$ListViewFn2@70bbbe4d, coder=KvCoder(BigEndianLongCoder,PCollectionViews$ValueOrMetadataCoder(FileBasedSink$FileResultCoder(NullableCoder(GlobalWindow$Coder)),OffsetRange$Coder)), windowMappingFn=GlobalWindowMappingFn{}, pCollection=null}
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4871)
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.get(IsmSideInputReader.java:336)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
Caused by: java.lang.IllegalArgumentException: Unknown type of view requested: SimplePCollectionView{tag=Tag<org.apache.beam.sdk.values.PCollectionViews$SimplePCollectionView.<init>:1261#1abfe6cb40bcf2e7>, viewFn=org.apache.beam.sdk.values.PCollectionViews$ListViewFn2@70bbbe4d, coder=KvCoder(BigEndianLongCoder,PCollectionViews$ValueOrMetadataCoder(FileBasedSink$FileResultCoder(NullableCoder(GlobalWindow$Coder)),OffsetRange$Coder)), windowMappingFn=GlobalWindowMappingFn{}, pCollection=null}
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.lambda$get$3(IsmSideInputReader.java:357)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4876)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4871)
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.get(IsmSideInputReader.java:336)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:232)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188)
at org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:339)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:212)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:163)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:92)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2021 11:27:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-29T11:27:54.406Z: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Unknown type of view requested: SimplePCollectionView{tag=Tag<org.apache.beam.sdk.values.PCollectionViews$SimplePCollectionView.<init>:1261#1abfe6cb40bcf2e7>, viewFn=org.apache.beam.sdk.values.PCollectionViews$ListViewFn2@70041151, coder=KvCoder(BigEndianLongCoder,PCollectionViews$ValueOrMetadataCoder(FileBasedSink$FileResultCoder(NullableCoder(GlobalWindow$Coder)),OffsetRange$Coder)), windowMappingFn=GlobalWindowMappingFn{}, pCollection=null}
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4871)
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.get(IsmSideInputReader.java:336)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
Caused by: java.lang.IllegalArgumentException: Unknown type of view requested: SimplePCollectionView{tag=Tag<org.apache.beam.sdk.values.PCollectionViews$SimplePCollectionView.<init>:1261#1abfe6cb40bcf2e7>, viewFn=org.apache.beam.sdk.values.PCollectionViews$ListViewFn2@70041151, coder=KvCoder(BigEndianLongCoder,PCollectionViews$ValueOrMetadataCoder(FileBasedSink$FileResultCoder(NullableCoder(GlobalWindow$Coder)),OffsetRange$Coder)), windowMappingFn=GlobalWindowMappingFn{}, pCollection=null}
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.lambda$get$3(IsmSideInputReader.java:357)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4876)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4871)
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.get(IsmSideInputReader.java:336)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:232)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188)
at org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:339)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:212)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:163)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:92)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2021 11:28:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-29T11:28:06.391Z: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Unknown type of view requested: SimplePCollectionView{tag=Tag<org.apache.beam.sdk.values.PCollectionViews$SimplePCollectionView.<init>:1261#1abfe6cb40bcf2e7>, viewFn=org.apache.beam.sdk.values.PCollectionViews$ListViewFn2@c09f6b7, coder=KvCoder(BigEndianLongCoder,PCollectionViews$ValueOrMetadataCoder(FileBasedSink$FileResultCoder(NullableCoder(GlobalWindow$Coder)),OffsetRange$Coder)), windowMappingFn=GlobalWindowMappingFn{}, pCollection=null}
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4871)
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.get(IsmSideInputReader.java:336)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
Caused by: java.lang.IllegalArgumentException: Unknown type of view requested: SimplePCollectionView{tag=Tag<org.apache.beam.sdk.values.PCollectionViews$SimplePCollectionView.<init>:1261#1abfe6cb40bcf2e7>, viewFn=org.apache.beam.sdk.values.PCollectionViews$ListViewFn2@c09f6b7, coder=KvCoder(BigEndianLongCoder,PCollectionViews$ValueOrMetadataCoder(FileBasedSink$FileResultCoder(NullableCoder(GlobalWindow$Coder)),OffsetRange$Coder)), windowMappingFn=GlobalWindowMappingFn{}, pCollection=null}
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.lambda$get$3(IsmSideInputReader.java:357)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4876)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4871)
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.get(IsmSideInputReader.java:336)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:232)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188)
at org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:339)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:212)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:163)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:92)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 29, 2021 11:28:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-29T11:28:06.441Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Apr 29, 2021 11:28:06 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-29T11:28:06.561Z: Workflow failed. Causes: S11:WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write failed., The job failed because a work item has failed 4 times. Look in previous log entries for the cause of each one of the 4 failures. For more information, see https://cloud.google.com/dataflow/docs/guides/common-errors. The work item was attempted on these workers:
userscore-jenkins-0429112-04290425-uq1f-harness-0d8c
Root cause: Work item failed.,
userscore-jenkins-0429112-04290425-uq1f-harness-0d8c
Root cause: Work item failed.,
userscore-jenkins-0429112-04290425-uq1f-harness-0d8c
Root cause: Work item failed.,
userscore-jenkins-0429112-04290425-uq1f-harness-0d8c
Root cause: Work item failed.
Apr 29, 2021 11:28:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-29T11:28:06.685Z: Cleaning up.
Apr 29, 2021 11:28:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-29T11:28:06.792Z: Stopping worker pool...
Apr 29, 2021 11:28:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-29T11:28:53.998Z: Autoscaling: Resized worker pool from 1 to 0.
Apr 29, 2021 11:28:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-29T11:28:54.048Z: Worker pool stopped.
Apr 29, 2021 11:29:00 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-04-29_04_25_32-5924452080346087329 failed with status FAILED.
gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 28m 41s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/q7q2jnhgpnxo2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1357
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1357/display/redirect?page=changes>
Changes:
[Ismaël Mejía] [BEAM-9283] Disable caching ValidatesRunner tests tasks
[Kenneth Knowles] Enable dual job submission for Dataflow
[Kyle Weaver] [BEAM-12009] Enable Calc splitting rule by default.
[zyichi] [BEAM-12045] Install google cloud profiler agent in java sdk container
[noreply] [BEAM-12016] Initial implementation for add_suffix, add_prefix (#14623)
[edre] Fix worker_pool plumbing to use a mutable List.
[noreply] [BEAM-12237] Fix logging in Samza Runner (#14657)
[Valentyn Tymofieiev] [BEAM-2085] Add resource hints support in Java SDK.
[Valentyn Tymofieiev] Translate Resource Hints into v1beta3 API in DF Runner and add test
[Valentyn Tymofieiev] Support passing resource hints through pipeline options.
[Valentyn Tymofieiev] Add resource hints supported by Dataflow.
[Ismaël Mejía] [BEAM-11055] Update log4j to version 2.14.1
------------------------------------------
[...truncated 3.74 MB...]
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.get(IsmSideInputReader.java:336)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:232)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188)
at org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:339)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:212)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:163)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:92)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 28, 2021 11:28:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-28T11:28:23.547Z: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Unknown type of view requested: SimplePCollectionView{tag=Tag<org.apache.beam.sdk.values.PCollectionViews$SimplePCollectionView.<init>:1261#1abfe6cb40bcf2e7>, viewFn=org.apache.beam.sdk.values.PCollectionViews$ListViewFn2@1edcd705, coder=KvCoder(BigEndianLongCoder,PCollectionViews$ValueOrMetadataCoder(FileBasedSink$FileResultCoder(NullableCoder(GlobalWindow$Coder)),OffsetRange$Coder)), windowMappingFn=GlobalWindowMappingFn{}, pCollection=null}
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4871)
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.get(IsmSideInputReader.java:336)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
Caused by: java.lang.IllegalArgumentException: Unknown type of view requested: SimplePCollectionView{tag=Tag<org.apache.beam.sdk.values.PCollectionViews$SimplePCollectionView.<init>:1261#1abfe6cb40bcf2e7>, viewFn=org.apache.beam.sdk.values.PCollectionViews$ListViewFn2@1edcd705, coder=KvCoder(BigEndianLongCoder,PCollectionViews$ValueOrMetadataCoder(FileBasedSink$FileResultCoder(NullableCoder(GlobalWindow$Coder)),OffsetRange$Coder)), windowMappingFn=GlobalWindowMappingFn{}, pCollection=null}
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.lambda$get$3(IsmSideInputReader.java:357)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4876)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4871)
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.get(IsmSideInputReader.java:336)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:232)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188)
at org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:339)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:212)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:163)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:92)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 28, 2021 11:28:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-28T11:28:36.037Z: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Unknown type of view requested: SimplePCollectionView{tag=Tag<org.apache.beam.sdk.values.PCollectionViews$SimplePCollectionView.<init>:1261#1abfe6cb40bcf2e7>, viewFn=org.apache.beam.sdk.values.PCollectionViews$ListViewFn2@40827db3, coder=KvCoder(BigEndianLongCoder,PCollectionViews$ValueOrMetadataCoder(FileBasedSink$FileResultCoder(NullableCoder(GlobalWindow$Coder)),OffsetRange$Coder)), windowMappingFn=GlobalWindowMappingFn{}, pCollection=null}
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4871)
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.get(IsmSideInputReader.java:336)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
Caused by: java.lang.IllegalArgumentException: Unknown type of view requested: SimplePCollectionView{tag=Tag<org.apache.beam.sdk.values.PCollectionViews$SimplePCollectionView.<init>:1261#1abfe6cb40bcf2e7>, viewFn=org.apache.beam.sdk.values.PCollectionViews$ListViewFn2@40827db3, coder=KvCoder(BigEndianLongCoder,PCollectionViews$ValueOrMetadataCoder(FileBasedSink$FileResultCoder(NullableCoder(GlobalWindow$Coder)),OffsetRange$Coder)), windowMappingFn=GlobalWindowMappingFn{}, pCollection=null}
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.lambda$get$3(IsmSideInputReader.java:357)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4876)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4871)
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.get(IsmSideInputReader.java:336)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:232)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188)
at org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:339)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:212)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:163)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:92)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 28, 2021 11:28:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-28T11:28:43.510Z: org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: Unknown type of view requested: SimplePCollectionView{tag=Tag<org.apache.beam.sdk.values.PCollectionViews$SimplePCollectionView.<init>:1261#1abfe6cb40bcf2e7>, viewFn=org.apache.beam.sdk.values.PCollectionViews$ListViewFn2@5fe99f57, coder=KvCoder(BigEndianLongCoder,PCollectionViews$ValueOrMetadataCoder(FileBasedSink$FileResultCoder(NullableCoder(GlobalWindow$Coder)),OffsetRange$Coder)), windowMappingFn=GlobalWindowMappingFn{}, pCollection=null}
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4871)
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.get(IsmSideInputReader.java:336)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
Caused by: java.lang.IllegalArgumentException: Unknown type of view requested: SimplePCollectionView{tag=Tag<org.apache.beam.sdk.values.PCollectionViews$SimplePCollectionView.<init>:1261#1abfe6cb40bcf2e7>, viewFn=org.apache.beam.sdk.values.PCollectionViews$ListViewFn2@5fe99f57, coder=KvCoder(BigEndianLongCoder,PCollectionViews$ValueOrMetadataCoder(FileBasedSink$FileResultCoder(NullableCoder(GlobalWindow$Coder)),OffsetRange$Coder)), windowMappingFn=GlobalWindowMappingFn{}, pCollection=null}
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.lambda$get$3(IsmSideInputReader.java:357)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4876)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4871)
at org.apache.beam.runners.dataflow.worker.IsmSideInputReader.get(IsmSideInputReader.java:336)
at org.apache.beam.runners.dataflow.worker.LazilyInitializedSideInputReader.get(LazilyInitializedSideInputReader.java:50)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.sideInput(SimpleDoFnRunner.java:267)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$1200(SimpleDoFnRunner.java:84)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.sideInput(SimpleDoFnRunner.java:395)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1.process(Reify.java:60)
at org.apache.beam.sdk.transforms.Reify$ReifyView$1$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:232)
at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188)
at org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:339)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:212)
at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:163)
at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:92)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 28, 2021 11:28:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-28T11:28:43.541Z: Finished operation WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Apr 28, 2021 11:28:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-28T11:28:43.645Z: Workflow failed. Causes: S11:WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write failed., The job failed because a work item has failed 4 times. Look in previous log entries for the cause of each one of the 4 failures. For more information, see https://cloud.google.com/dataflow/docs/guides/common-errors. The work item was attempted on these workers:
userscore-jenkins-0428112-04280426-24s1-harness-dbsl
Root cause: Work item failed.,
userscore-jenkins-0428112-04280426-24s1-harness-dbsl
Root cause: Work item failed.,
userscore-jenkins-0428112-04280426-24s1-harness-dbsl
Root cause: Work item failed.,
userscore-jenkins-0428112-04280426-24s1-harness-dbsl
Root cause: Work item failed.
Apr 28, 2021 11:28:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-28T11:28:43.721Z: Cleaning up.
Apr 28, 2021 11:28:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-28T11:28:43.811Z: Stopping worker pool...
Apr 28, 2021 11:29:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-28T11:29:26.395Z: Autoscaling: Resized worker pool from 1 to 0.
Apr 28, 2021 11:29:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-28T11:29:26.422Z: Worker pool stopped.
Apr 28, 2021 11:29:37 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-04-28_04_26_13-14474808158442233521 failed with status FAILED.
gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:direct-java:runMobileGamingJavaDirect'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 29m 19s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/gdufxeanfnf26
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1356
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1356/display/redirect?page=changes>
Changes:
[ajamato] [BEAM-11994] Refactor BigQueryTornadoes to make more options
[zyichi] [BEAM-12221] Add experimental pipeline option to control the execution
[Kenneth Knowles] Add View.toList pipeline to translator tests
[Kenneth Knowles] Temporarily disable FnApiDoFnRunnerTest.testUsingMetrics
[Kenneth Knowles] Resolve beam_fn_api experiment for side inputs, using portable expansion
------------------------------------------
[...truncated 4.43 MB...]
Apr 27, 2021 9:10:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/StreamingWrite/BatchedStreamingWrite.ViaBundleFinalization/ParMultiDo(BatchAndInsertElements) as step s18
Apr 27, 2021 9:10:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CalculateUserScores/LeaderboardUserGlobalWindow/Window.Assign as step s19
Apr 27, 2021 9:10:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CalculateUserScores/ExtractUserScore/MapElements/Map as step s20
Apr 27, 2021 9:10:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CalculateUserScores/ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey as step s21
Apr 27, 2021 9:10:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding CalculateUserScores/ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues as step s22
Apr 27, 2021 9:10:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WriteUserScoreSums/ConvertToRow as step s23
Apr 27, 2021 9:10:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WriteUserScoreSums/BigQueryIO.Write/PrepareWrite/ParDo(Anonymous) as step s24
Apr 27, 2021 9:10:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/CreateTables/ParDo(CreateTables) as step s25
Apr 27, 2021 9:10:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/ShardTableWrites as step s26
Apr 27, 2021 9:10:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/TagWithUniqueIds as step s27
Apr 27, 2021 9:10:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/Window.Into()/Window.Assign as step s28
Apr 27, 2021 9:10:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey as step s29
Apr 27, 2021 9:10:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/ExpandIterable as step s30
Apr 27, 2021 9:10:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/GlobalWindow/Window.Assign as step s31
Apr 27, 2021 9:10:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/StripShardId/Map as step s32
Apr 27, 2021 9:10:06 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/StreamingWrite/BatchedStreamingWrite.ViaBundleFinalization/ParMultiDo(BatchAndInsertElements) as step s33
Apr 27, 2021 9:10:07 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.30.0-SNAPSHOT
Apr 27, 2021 9:10:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2021-04-27_14_10_07-12002701699047671668?project=apache-beam-testing
Apr 27, 2021 9:10:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2021-04-27_14_10_07-12002701699047671668
Apr 27, 2021 9:10:08 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-04-27_14_10_07-12002701699047671668
Apr 27, 2021 9:10:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2021-04-27T21:10:10.479Z: Autoscaling is enabled for Dataflow Streaming Engine. Workers will scale between 1 and 100 unless maxNumWorkers is specified.
Apr 27, 2021 9:10:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:10.997Z: Autoscaling is enabled for job 2021-04-27_14_10_07-12002701699047671668. The number of workers will be between 1 and 100.
Apr 27, 2021 9:10:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:11.079Z: Autoscaling was automatically enabled for job 2021-04-27_14_10_07-12002701699047671668.
Apr 27, 2021 9:10:19 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:18.383Z: Worker configuration: n1-standard-2 in us-central1-f.
Apr 27, 2021 9:10:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:19.447Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 27, 2021 9:10:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:20.303Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Apr 27, 2021 9:10:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:20.341Z: Expanding GroupByKey operations into streaming Read/Write steps
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:20.554Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:20.815Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:20.853Z: Fusing consumer CalculateUserScores/LeaderboardUserGlobalWindow/Window.Assign into ParseGameEvent
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:20.897Z: Fusing consumer CalculateTeamScores/LeaderboardTeamFixedWindows/Window.Assign into ParseGameEvent
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:20.932Z: Fusing consumer PubsubIO.Read/MapElements/Map into PubsubIO.Read/PubsubUnboundedSource
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:20.964Z: Fusing consumer ParseGameEvent into PubsubIO.Read/MapElements/Map
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.012Z: Fusing consumer CalculateTeamScores/ExtractTeamScore/MapElements/Map into CalculateTeamScores/LeaderboardTeamFixedWindows/Window.Assign
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.088Z: Fusing consumer CalculateTeamScores/ExtractTeamScore/Combine.perKey(SumInteger)/GroupByKey/WriteStream into CalculateTeamScores/ExtractTeamScore/MapElements/Map
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.129Z: Fusing consumer CalculateTeamScores/ExtractTeamScore/Combine.perKey(SumInteger)/Combine.GroupedValues into CalculateTeamScores/ExtractTeamScore/Combine.perKey(SumInteger)/GroupByKey/ReadStream
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.164Z: Fusing consumer WriteTeamScoreSums/ConvertToRow into CalculateTeamScores/ExtractTeamScore/Combine.perKey(SumInteger)/Combine.GroupedValues
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.199Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/PrepareWrite/ParDo(Anonymous) into WriteTeamScoreSums/ConvertToRow
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.240Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/CreateTables/ParDo(CreateTables) into WriteTeamScoreSums/BigQueryIO.Write/PrepareWrite/ParDo(Anonymous)
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.268Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/ShardTableWrites into WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/CreateTables/ParDo(CreateTables)
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.311Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/TagWithUniqueIds into WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/ShardTableWrites
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.346Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/Window.Into()/Window.Assign into WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/TagWithUniqueIds
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.389Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/WriteStream into WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/Window.Into()/Window.Assign
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.437Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/MergeBuckets into WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/ReadStream
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.481Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/ExpandIterable into WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/MergeBuckets
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.525Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/GlobalWindow/Window.Assign into WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/ExpandIterable
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.557Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/StripShardId/Map into WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/GlobalWindow/Window.Assign
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.596Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/StreamingWrite/BatchedStreamingWrite.ViaBundleFinalization/ParMultiDo(BatchAndInsertElements) into WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/StripShardId/Map
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.629Z: Fusing consumer CalculateUserScores/ExtractUserScore/MapElements/Map into CalculateUserScores/LeaderboardUserGlobalWindow/Window.Assign
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.663Z: Fusing consumer CalculateUserScores/ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/WriteStream into CalculateUserScores/ExtractUserScore/MapElements/Map
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.689Z: Fusing consumer CalculateUserScores/ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues into CalculateUserScores/ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/ReadStream
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.725Z: Fusing consumer WriteUserScoreSums/ConvertToRow into CalculateUserScores/ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.761Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/PrepareWrite/ParDo(Anonymous) into WriteUserScoreSums/ConvertToRow
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.797Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/CreateTables/ParDo(CreateTables) into WriteUserScoreSums/BigQueryIO.Write/PrepareWrite/ParDo(Anonymous)
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.823Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/ShardTableWrites into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/CreateTables/ParDo(CreateTables)
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.887Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/TagWithUniqueIds into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/ShardTableWrites
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.928Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/Window.Into()/Window.Assign into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/TagWithUniqueIds
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.961Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/WriteStream into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/Window.Into()/Window.Assign
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:21.999Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/MergeBuckets into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/ReadStream
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:22.083Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/ExpandIterable into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/MergeBuckets
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:22.122Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/GlobalWindow/Window.Assign into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/ExpandIterable
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:22.184Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/StripShardId/Map into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/GlobalWindow/Window.Assign
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:22.247Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/StreamingWrite/BatchedStreamingWrite.ViaBundleFinalization/ParMultiDo(BatchAndInsertElements) into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/StripShardId/Map
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:22.314Z: The pubsub read for: projects/apache-beam-testing/subscriptions/java_mobile_gaming_topic.subscription-7166221311757774785 is configured to compute input data watermarks based on custom timestamp attribute timestamp_ms. Cloud Dataflow has created an additional tracking subscription to do this, which will be cleaned up automatically. For details, see: https://cloud.google.com/dataflow/model/pubsub-io#timestamps-ids
Apr 27, 2021 9:10:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:22.623Z: Starting 1 workers in us-central1-f...
Apr 27, 2021 9:10:26 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:25.079Z: Pub/Sub resources set up for topic 'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Apr 27, 2021 9:10:43 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:10:42.453Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 27, 2021 9:11:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:11:15.966Z: Autoscaling: Raised the number of workers to 1 so that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2021 9:11:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:11:52.864Z: Workers have started successfully.
Apr 27, 2021 9:11:53 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T21:11:52.921Z: Workers have started successfully.
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project word-count-beam: An exception occured while executing the Java class. unable to create new native thread -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow FAILED
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom
Repository URL: https://repository.apache.org/content/repositories/snapshots
Version: 2.30.0-SNAPSHOT
GCS Project: apache-beam-testing
GCS Region: us-central1
GCS Storage bucket: temp-storage-for-release-validation-tests/nightly-snapshot-validation
BigQuery Dataset: beam_postrelease_mobile_gaming
PubSub Topic: java_mobile_gaming_topic
**************************************
* Scenario: Run Apache Beam Java SDK Mobile Gaming Examples using GCP BOM - Dataflow
**************************************
mvn archetype:generate --update-snapshots -DarchetypeGroupId=org.apache.beam -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-gcp-bom-examples -DarchetypeVersion=2.30.0-SNAPSHOT -DgroupId=org.example -DartifactId=word-count-beam -Dversion="0.1" -Dpackage=org.apache.beam.examples -DinteractiveMode=false
Using maven /home/jenkins/tools/maven/apache-maven-3.5.4
Error occurred during initialization of VM
java.lang.OutOfMemoryError: unable to create new native thread
Error occurred during initialization of VM
java.lang.OutOfMemoryError: unable to create new native thread
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom FAILED
> Task :runners:google-cloud-dataflow-java:runQuickstartJavaDataflow
Repository URL: https://repository.apache.org/content/repositories/snapshots
Version: 2.30.0-SNAPSHOT
GCS Project: apache-beam-testing
GCS Region: us-central1
GCS Storage bucket: temp-storage-for-release-validation-tests/nightly-snapshot-validation
**************************************
* Scenario: Run Apache Beam Java SDK Quickstart - Dataflow
**************************************
**************************************
* Test: Gets the WordCount Example Code
**************************************
mvn archetype:generate --update-snapshots -DarchetypeGroupId=org.apache.beam -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples -DarchetypeVersion=2.30.0-SNAPSHOT -DgroupId=org.example -DartifactId=word-count-beam -Dversion="0.1" -Dpackage=org.apache.beam.examples -DinteractiveMode=false
Using maven /home/jenkins/tools/maven/apache-maven-3.5.4
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create GC thread. Out of system resources.
# An error report file with more information is saved as:
# /tmp/groovy-generated-2032419933197306368-tmpdir/hs_err_pid17316.log
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create GC thread. Out of system resources.
# An error report file with more information is saved as:
# /tmp/groovy-generated-2032419933197306368-tmpdir/hs_err_pid17316.log
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runQuickstartJavaDataflow FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 37m 55s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/je352xpk4b67w
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1355
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1355/display/redirect?page=changes>
Changes:
[Sylvain Wallez] [BEAM-12226] JdbcIO DefaultRetryStrategy: retry on PostgreSQL-specific
[Kenneth Knowles] [BEAM-12231] Update Dataflow container to adjust for API changes
[noreply] Clarify when to use `@DefaultCoder` (#14326)
[noreply] [BEAM-12232] Expose public factory methods in SparkJobServerDriver
------------------------------------------
[...truncated 3.98 MB...]
INFO: 2021-04-27T20:12:51.372Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/StreamingWrite/BatchedStreamingWrite.ViaBundleFinalization/ParMultiDo(BatchAndInsertElements) into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/StripShardId/Map
Apr 27, 2021 8:12:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T20:12:51.426Z: The pubsub read for: projects/apache-beam-testing/subscriptions/java_mobile_gaming_topic.subscription-8620046984745911465 is configured to compute input data watermarks based on custom timestamp attribute timestamp_ms. Cloud Dataflow has created an additional tracking subscription to do this, which will be cleaned up automatically. For details, see: https://cloud.google.com/dataflow/model/pubsub-io#timestamps-ids
Apr 27, 2021 8:12:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T20:12:51.716Z: Starting 1 workers in us-central1-f...
Apr 27, 2021 8:12:55 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T20:12:54.500Z: Pub/Sub resources set up for topic 'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Apr 27, 2021 8:13:08 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T20:13:06.068Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r4339b9a0c45f5b96_0000017914f72101_1 ... (0s) Current status: RUNNING
Waiting on bqjob_r4339b9a0c45f5b96_0000017914f72101_1 ... (0s) Current status: DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Apr 27, 2021 8:13:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T20:13:37.945Z: Autoscaling: Raised the number of workers to 1 so that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2021 8:14:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T20:14:07.950Z: Workers have started successfully.
Apr 27, 2021 8:14:09 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T20:14:07.986Z: Workers have started successfully.
..............................................................................................................................................................................................................Introducing a parse error.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r66567297883299f2_0000017914f8192e_1 ... (0s) Current status: RUNNING
Waiting on bqjob_r66567297883299f2_0000017914f8192e_1 ... (0s) Current status: DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r4eb6a67b6add036_0000017914f91168_1 ... (0s) Current status: RUNNING
Waiting on bqjob_r4eb6a67b6add036_0000017914f91168_1 ... (0s) Current status: DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r317d9cdf4ca0cea6_0000017914fa0995_1 ... (0s) Current status: RUNNING
Waiting on bqjob_r317d9cdf4ca0cea6_0000017914fa0995_1 ... (0s) Current status: DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
.........................................................................................................................................................................................................................................................................................................................................................................................................DELAY(490944, 1)
{timestamp_ms=1619554158000}
late data for: user17_AzureKookaburra,AzureKookaburra,6,1619554158000,2021-04-27 13:17:29.859
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r53e5e0fd4b1829e_0000017914fb0090_1 ... (0s) Current status: RUNNING
Waiting on bqjob_r53e5e0fd4b1829e_0000017914fb0090_1 ... (1s) Current status: RUNNING
Waiting on bqjob_r53e5e0fd4b1829e_0000017914fb0090_1 ... (1s) Current status: DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r20b2e250bd532555_0000017914fbfc89_1 ... (0s) Current status: RUNNING
Waiting on bqjob_r20b2e250bd532555_0000017914fbfc89_1 ... (0s) Current status: DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r4edfd412fbfafcb0_0000017914fcf457_1 ... (0s) Current status: RUNNING
Waiting on bqjob_r4edfd412fbfafcb0_0000017914fcf457_1 ... (0s) Current status: DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
.........................................................................................................................................................................................................................................................................................................................Introducing a parse error.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r27394325eed02807_0000017914fdeb8f_1 ... (0s) Current status: RUNNING
Waiting on bqjob_r27394325eed02807_0000017914fdeb8f_1 ... (0s) Current status: DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r3ccb8b6637cf8b9c_0000017914fee348_1 ... (0s) Current status: RUNNING
Waiting on bqjob_r3ccb8b6637cf8b9c_0000017914fee348_1 ... (0s) Current status: DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
..............................................................................................................................................................................................................................................................................................DELAY(346122, 1)
{timestamp_ms=1619554604000}
late data for: user3_AmethystWallaby,AmethystWallaby,5,1619554604000,2021-04-27 13:22:30.468
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r52fa049f110abbb1_0000017914ffdc04_1 ... (0s) Current status: RUNNING
Waiting on bqjob_r52fa049f110abbb1_0000017914ffdc04_1 ... (0s) Current status: DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r3544b20e1fbd0100_000001791500d71a_1 ... (0s) Current status: RUNNING
Waiting on bqjob_r3544b20e1fbd0100_000001791500d71a_1 ... (0s) Current status: DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r758b7793fff132a2_000001791501ce6e_1 ... (0s) Current status: RUNNING
Waiting on bqjob_r758b7793fff132a2_000001791501ce6e_1 ... (0s) Current status: DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DataflowRunner_user |
| leaderboard_DirectRunner_team |
+-----------------------------------+
bq query --batch "SELECT user FROM [apache-beam-testing:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_user] LIMIT 10"
Waiting on bqjob_r40d4e689ac3027b7_000001791501dcee_1 ... (0s) Current status: RUNNING
Waiting on bqjob_r40d4e689ac3027b7_000001791501dcee_1 ... (0s) Current status: DONE
+------------------------------+
| user |
+------------------------------+
| user2_BattleshipGreyPlatypus |
| user6_BananaEmu |
| user5_AuburnWombat |
| user4_AzureKookaburra |
| user3_RubyBandicoot |
| user1_AlmondQuokka |
| user7_AuburnPlatypus |
| user0_AzureEchidna |
| user0_AuburnNumbat |
| user5_BattleshipGreyDingo |
+------------------------------+
Verified Almond
RUNNING_JOB=`gcloud dataflow jobs list | grep leaderboard-validation-1619554344230-469 | grep Running | cut -d' ' -f1`
if [ ! -z "${RUNNING_JOB}" ]
then
gcloud dataflow jobs cancel ${RUNNING_JOB}
else
echo "Job 'leaderboard-validation-1619554344230-469' is not running."
fi
Cancelled job [2021-04-27_13_12_38-10244767523801253557]
**************************************
* SUCCESS: LeaderBoard successfully run on DataflowRunner. with Streaming Engine
**************************************
[SUCCESS]
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:direct-java:runMobileGamingJavaDirect'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 35m 17s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/ghgqftmelejt6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1354
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1354/display/redirect?page=changes>
Changes:
[daniel.o.programmer] [BEAM-11418] Fix Go universal runner x-lang artifact staging.
[Kyle Weaver] [BEAM-12219] Enable SUBSTRING function.
[Brian Hulette] Close ZipOutputStream
[chamikaramj] 'sdk_harness_container_image_overrides' works when there are multiple
[aromanenko.dev] [BEAM-12214] RedisIO.readAll() was removed since it was deprecated since
[aromanenko.dev] [BEAM-12216] MqttIO.create() with clientId constructor removed because
[noreply] [BEAM-9582] Bump grpcio-tools to the latest version (#14576)
[noreply] [BEAM-3713] Moving integration tests from nose to pytest (#14481)
[noreply] [BEAM-10029] Adds Grafana config for Spanner IO Perf Metrics (#14643)
------------------------------------------
[...truncated 4.58 MB...]
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-04-27_04_28_30-18148439122759749696
Apr 27, 2021 11:28:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:33.833Z: Autoscaling is enabled for job 2021-04-27_04_28_30-18148439122759749696. The number of workers will be between 1 and 1000.
Apr 27, 2021 11:28:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:33.898Z: Autoscaling was automatically enabled for job 2021-04-27_04_28_30-18148439122759749696.
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:40.917Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:41.576Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:41.781Z: Expanding GroupByKey operations into optimizable parts.
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:41.815Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:41.981Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.009Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.031Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.059Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.098Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.144Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.178Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.210Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.254Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.279Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.310Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.361Z: Unzipping flatten s12 for input s11.org.apache.beam.sdk.values.PCollection.<init>:402#5f2ef1f005ae0b4
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.397Z: Fusing unzipped copy of WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.428Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.509Z: Fusing consumer ParseGameEvent into TextIO.Read/Read
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.541Z: Fusing consumer ExtractUserScore/MapElements/Map into ParseGameEvent
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.576Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial into ExtractUserScore/MapElements/Map
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.599Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.625Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.646Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.697Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.733Z: Fusing consumer WriteUserScoreSums/ConvertToRow into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.767Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign into WriteUserScoreSums/ConvertToRow
Apr 27, 2021 11:28:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.793Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign
Apr 27, 2021 11:28:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.817Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 27, 2021 11:28:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.852Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
Apr 27, 2021 11:28:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.888Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
Apr 27, 2021 11:28:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.919Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
Apr 27, 2021 11:28:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:42.952Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
Apr 27, 2021 11:28:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:43.458Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 27, 2021 11:28:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:43.515Z: Starting 1 workers in us-central1-f...
Apr 27, 2021 11:28:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:43.746Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 27, 2021 11:28:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:43.925Z: Executing operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 27, 2021 11:28:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:28:58.422Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 27, 2021 11:29:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:29:42.210Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 27, 2021 11:30:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:30:28.906Z: Workers have started successfully.
Apr 27, 2021 11:30:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:30:28.943Z: Workers have started successfully.
Apr 27, 2021 11:30:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-27T11:30:38.638Z: java.lang.NoClassDefFoundError: org/apache/beam/sdk/metrics/MetricsLogger
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.metrics.MetricsLogger
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 28 more
Apr 27, 2021 11:30:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-27T11:30:46.089Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 27, 2021 11:30:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-27T11:30:57.317Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 27, 2021 11:31:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-27T11:31:16.043Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 27, 2021 11:31:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:31:16.068Z: Finished operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 27, 2021 11:31:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-27T11:31:16.220Z: Workflow failed. Causes: S02:TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write failed., Internal Issue (eae55ac8a270ca39): 63963027:24514
Apr 27, 2021 11:31:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:31:16.308Z: Cleaning up.
Apr 27, 2021 11:31:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:31:16.432Z: Stopping worker pool...
Apr 27, 2021 11:32:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:32:13.813Z: Autoscaling: Resized worker pool from 1 to 0.
Apr 27, 2021 11:32:15 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-27T11:32:13.850Z: Worker pool stopped.
Apr 27, 2021 11:32:27 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-04-27_04_28_30-18148439122759749696 failed with status FAILED.
gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 32m 11s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/btsqof2nkacyg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1353
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1353/display/redirect?page=changes>
Changes:
[Ismaël Mejía] [BEAM-7237] Refine SparkRunner exception management to fix
------------------------------------------
[...truncated 3.82 MB...]
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-04-26_04_28_43-9734476955075135377
Apr 26, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:46.721Z: Autoscaling is enabled for job 2021-04-26_04_28_43-9734476955075135377. The number of workers will be between 1 and 1000.
Apr 26, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:46.823Z: Autoscaling was automatically enabled for job 2021-04-26_04_28_43-9734476955075135377.
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:53.807Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:54.693Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:54.852Z: Expanding GroupByKey operations into optimizable parts.
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:54.893Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.071Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.105Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.134Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.162Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.188Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.218Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.254Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.291Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.331Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.357Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.385Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.440Z: Unzipping flatten s12 for input s11.org.apache.beam.sdk.values.PCollection.<init>:402#5f2ef1f005ae0b4
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.477Z: Fusing unzipped copy of WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.523Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.549Z: Fusing consumer ParseGameEvent into TextIO.Read/Read
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.590Z: Fusing consumer ExtractUserScore/MapElements/Map into ParseGameEvent
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.624Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial into ExtractUserScore/MapElements/Map
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.657Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.702Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.732Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.765Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.810Z: Fusing consumer WriteUserScoreSums/ConvertToRow into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.836Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign into WriteUserScoreSums/ConvertToRow
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.862Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.896Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.925Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.960Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:55.984Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:56.009Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
Apr 26, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:56.403Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 26, 2021 11:28:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:56.484Z: Starting 1 workers in us-central1-c...
Apr 26, 2021 11:28:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:56.967Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 26, 2021 11:28:57 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:28:57.114Z: Executing operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 26, 2021 11:29:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:29:04.933Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 26, 2021 11:29:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:29:49.661Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 26, 2021 11:30:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:30:16.372Z: Workers have started successfully.
Apr 26, 2021 11:30:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:30:16.407Z: Workers have started successfully.
Apr 26, 2021 11:30:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-26T11:30:24.792Z: java.lang.NoClassDefFoundError: org/apache/beam/sdk/metrics/MetricsLogger
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.metrics.MetricsLogger
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 28 more
Apr 26, 2021 11:30:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-26T11:30:32.056Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 26, 2021 11:30:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-26T11:30:41.793Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 26, 2021 11:30:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-26T11:30:55.758Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 26, 2021 11:30:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:30:55.784Z: Finished operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 26, 2021 11:30:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-26T11:30:55.893Z: Workflow failed. Causes: S02:TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write failed., Internal Issue (a31c2b922054f6e5): 63963027:24514
Apr 26, 2021 11:30:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:30:55.960Z: Cleaning up.
Apr 26, 2021 11:30:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:30:56.047Z: Stopping worker pool...
Apr 26, 2021 11:31:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:31:47.298Z: Autoscaling: Resized worker pool from 1 to 0.
Apr 26, 2021 11:31:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-26T11:31:47.340Z: Worker pool stopped.
Apr 26, 2021 11:31:55 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-04-26_04_28_43-9734476955075135377 failed with status FAILED.
gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 31m 37s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/wabhmhuu3ajju
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1352
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1352/display/redirect?page=changes>
Changes:
[Boyuan Zhang] Remove runner V2 overrides from V1 codepath
[noreply] Merge pull request #14591: [BEAM-10277][BEAM-12198] Dataflow update
------------------------------------------
[...truncated 3.81 MB...]
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:54.338Z: Expanding GroupByKey operations into optimizable parts.
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:54.357Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:54.547Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:54.586Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:54.618Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:54.640Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:54.665Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:54.709Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:54.740Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:54.775Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:54.809Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:54.833Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:54.861Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:54.897Z: Unzipping flatten s12 for input s11.org.apache.beam.sdk.values.PCollection.<init>:402#5f2ef1f005ae0b4
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:54.929Z: Fusing unzipped copy of WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:54.965Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:54.997Z: Fusing consumer ParseGameEvent into TextIO.Read/Read
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:55.020Z: Fusing consumer ExtractUserScore/MapElements/Map into ParseGameEvent
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:55.056Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial into ExtractUserScore/MapElements/Map
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:55.087Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:55.122Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:55.155Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:55.187Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:55.218Z: Fusing consumer WriteUserScoreSums/ConvertToRow into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:55.245Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign into WriteUserScoreSums/ConvertToRow
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:55.277Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:55.308Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:55.334Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:55.362Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:55.393Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:55.410Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:55.788Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:55.868Z: Starting 1 workers in us-central1-c...
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:56.197Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 25, 2021 11:28:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:28:56.326Z: Executing operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 25, 2021 11:29:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:29:09.884Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 25, 2021 11:29:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:29:47.083Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 25, 2021 11:30:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:30:13.071Z: Workers have started successfully.
Apr 25, 2021 11:30:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:30:13.166Z: Workers have started successfully.
Apr 25, 2021 11:30:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-25T11:30:23.922Z: java.lang.NoClassDefFoundError: org/apache/beam/sdk/metrics/MetricsLogger
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.metrics.MetricsLogger
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 28 more
Apr 25, 2021 11:30:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-25T11:30:32.276Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 25, 2021 11:30:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-25T11:30:39.042Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 25, 2021 11:30:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-25T11:30:53.479Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 25, 2021 11:30:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:30:53.507Z: Finished operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 25, 2021 11:30:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-25T11:30:53.731Z: Workflow failed. Causes: S02:TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write failed., Internal Issue (4cb490dc15e65035): 63963027:24514
Apr 25, 2021 11:30:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:30:53.797Z: Cleaning up.
Apr 25, 2021 11:30:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:30:53.886Z: Stopping worker pool...
Apr 25, 2021 11:31:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:31:37.054Z: Autoscaling: Resized worker pool from 1 to 0.
Apr 25, 2021 11:31:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-25T11:31:37.092Z: Worker pool stopped.
Apr 25, 2021 11:31:43 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-04-25_04_28_40-2690716321286251779 failed with status FAILED.
gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:twister2:runQuickstartJavaTwister2'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 31m 25s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/3qok5axqukifs
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1351
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1351/display/redirect?page=changes>
Changes:
[Boyuan Zhang] [BEAM-12193] Add user metrics to show founded TopicPartition
[yifanmai] Update annotations in ValidateRunner tests for pack_combiners
[yifanmai] Run pack_combiners phase by default in DataflowRunner
[yifanmai] Fix tests
[yifanmai] Fix tests
[yifanmai] Lint
[yifanmai] Lint
[yoshiki.obata] [BEAM-7372] cleanup py2 codepath from apache_beam/runners/direct, job,
[suztomo] [BEAM-8357] Upgrading auto-value to 1.8.1 from 1.8
[mackenzieclark] Fixing counter names from PR https://github.com/apache/beam/pull/14233
[aromanenko.dev] [BEAM-12217] MongoDbIO: Read.withFilter() and Read.withProjection() are
[noreply] [BEAM-12209] Fix thread safety of DirectStreamObserver by using atomic
[noreply] [BEAM-12127] More optimizations for PCollectionConsumerRegistry: cache
[noreply] [BEAM-12204] Allow unboundedly large side inputs in portable Java.
[Boyuan Zhang] [BEAM-12192] Have WatchKafkaTopicPartitionDoFn respects topics given by
[noreply] [BEAM-10937] Tour of Beam use FileSystems for I/O (#14431)
------------------------------------------
[...truncated 3.81 MB...]
Apr 24, 2021 11:28:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:45.778Z: Expanding GroupByKey operations into optimizable parts.
Apr 24, 2021 11:28:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:45.836Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 24, 2021 11:28:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.026Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 24, 2021 11:28:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.079Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
Apr 24, 2021 11:28:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.112Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
Apr 24, 2021 11:28:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.155Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
Apr 24, 2021 11:28:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.194Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.237Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.282Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.320Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.355Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.390Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.415Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.458Z: Unzipping flatten s12 for input s11.org.apache.beam.sdk.values.PCollection.<init>:402#5f2ef1f005ae0b4
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.502Z: Fusing unzipped copy of WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.546Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.591Z: Fusing consumer ParseGameEvent into TextIO.Read/Read
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.625Z: Fusing consumer ExtractUserScore/MapElements/Map into ParseGameEvent
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.672Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial into ExtractUserScore/MapElements/Map
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.713Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.751Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.782Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.815Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.857Z: Fusing consumer WriteUserScoreSums/ConvertToRow into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.892Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign into WriteUserScoreSums/ConvertToRow
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.926Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:46.968Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:47.000Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:47.032Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:47.086Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:47.128Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:47.810Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:47.903Z: Starting 1 workers in us-central1-f...
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:48.319Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 24, 2021 11:28:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:48.475Z: Executing operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 24, 2021 11:28:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:28:59.245Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 24, 2021 11:29:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:29:34.070Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 24, 2021 11:30:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:30:10.449Z: Workers have started successfully.
Apr 24, 2021 11:30:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:30:10.493Z: Workers have started successfully.
Apr 24, 2021 11:30:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-24T11:30:19.057Z: java.lang.NoClassDefFoundError: org/apache/beam/sdk/metrics/MetricsLogger
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.metrics.MetricsLogger
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 28 more
Apr 24, 2021 11:30:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-24T11:30:26.139Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 24, 2021 11:30:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-24T11:30:33.775Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 24, 2021 11:30:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-24T11:30:52.478Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 24, 2021 11:30:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:30:52.507Z: Finished operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 24, 2021 11:30:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-24T11:30:52.672Z: Workflow failed. Causes: S02:TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write failed., Internal Issue (97ead4341592b8b9): 63963027:24514
Apr 24, 2021 11:30:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:30:52.769Z: Cleaning up.
Apr 24, 2021 11:30:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:30:52.884Z: Stopping worker pool...
Apr 24, 2021 11:31:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:31:35.702Z: Autoscaling: Resized worker pool from 1 to 0.
Apr 24, 2021 11:31:35 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-24T11:31:35.751Z: Worker pool stopped.
Apr 24, 2021 11:31:49 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-04-24_04_28_35-16510524361828445964 failed with status FAILED.
gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:direct-java:runMobileGamingJavaDirect'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 31m 32s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/b6hkfhlzs7np4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1350
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1350/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-12194] Implement CREATE AGGREGATE FUNCTION.
[samuelw] [BEAM-12203] Reduce thread switches and queuing for BeamFnControlClient
[Daniel Kulp] [BEAM-12210] Use formatting string for checkArgument to avoid excess
[Daniel Kulp] Fix spotless
[noreply] [BEAM-8691] Upgrading bigtable-client-core to 1.19.1
[emilyye] add sdkContainerImage to Java WorkerPool PipelineOptions
[suztomo] [BEAM-8684] Upgrading com.google.apis:google-api-services-bigquery
[suztomo] [BEAM-8750] Upgrading com.google.apis:google-api-services-clouddebugger
[suztomo] [BEAM-8751] Upgrading
[suztomo] [BEAM-8752] Upgrading com.google.apis:google-api-services-dataflow
[suztomo] [BEAM-10349] Upgrading com.google.apis:google-api-services-healthcare
[suztomo] [BEAM-8753] Upgrading com.google.apis:google-api-services-pubsub
[noreply] Plumb worker pool from properties to capabilities in Dataflow java
------------------------------------------
[...truncated 3.76 MB...]
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-04-23_04_28_07-14158877526880069398
Apr 23, 2021 11:28:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:11.109Z: Autoscaling is enabled for job 2021-04-23_04_28_07-14158877526880069398. The number of workers will be between 1 and 1000.
Apr 23, 2021 11:28:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:11.457Z: Autoscaling was automatically enabled for job 2021-04-23_04_28_07-14158877526880069398.
Apr 23, 2021 11:28:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:19.046Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:19.686Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:19.824Z: Expanding GroupByKey operations into optimizable parts.
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:19.915Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.119Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.151Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.188Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.217Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.252Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.283Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.308Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.335Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.369Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.401Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.425Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.452Z: Unzipping flatten s12 for input s11.org.apache.beam.sdk.values.PCollection.<init>:402#5f2ef1f005ae0b4
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.488Z: Fusing unzipped copy of WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.524Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.567Z: Fusing consumer ParseGameEvent into TextIO.Read/Read
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.597Z: Fusing consumer ExtractUserScore/MapElements/Map into ParseGameEvent
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.633Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial into ExtractUserScore/MapElements/Map
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.666Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.687Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.721Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.751Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.787Z: Fusing consumer WriteUserScoreSums/ConvertToRow into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.813Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign into WriteUserScoreSums/ConvertToRow
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.845Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.910Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.947Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:20.976Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:21.010Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:21.046Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:21.493Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 23, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:21.601Z: Starting 1 workers in us-central1-c...
Apr 23, 2021 11:28:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:22.055Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 23, 2021 11:28:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:22.222Z: Executing operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 23, 2021 11:28:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:49.473Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 23, 2021 11:28:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:28:58.357Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 23, 2021 11:29:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:29:45.317Z: Workers have started successfully.
Apr 23, 2021 11:29:46 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:29:45.348Z: Workers have started successfully.
Apr 23, 2021 11:29:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-23T11:29:53.928Z: java.lang.NoClassDefFoundError: org/apache/beam/sdk/metrics/MetricsLogger
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.metrics.MetricsLogger
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 28 more
Apr 23, 2021 11:30:04 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-23T11:30:02.864Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 23, 2021 11:30:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-23T11:30:11.386Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 23, 2021 11:30:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-23T11:30:30.325Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 23, 2021 11:30:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:30:30.398Z: Finished operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 23, 2021 11:30:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-23T11:30:30.643Z: Workflow failed. Causes: S02:TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write failed., Internal Issue (3075ee76cd5ff953): 63963027:24514
Apr 23, 2021 11:30:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:30:30.858Z: Cleaning up.
Apr 23, 2021 11:30:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:30:31.218Z: Stopping worker pool...
Apr 23, 2021 11:31:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:31:20.013Z: Autoscaling: Resized worker pool from 1 to 0.
Apr 23, 2021 11:31:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-23T11:31:20.089Z: Worker pool stopped.
Apr 23, 2021 11:31:28 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-04-23_04_28_07-14158877526880069398 failed with status FAILED.
gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 31m 10s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/kb7at27ovjwsg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1349
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1349/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-12009] Copy CalcRelSplitter.
[Kyle Weaver] [BEAM-12009] Implement Calc splitting rule.
[Kyle Weaver] [BEAM-12009] Use different method signature to avoid spurious null check
[Kyle Weaver] [BEAM-12009] Move CalcRelSplitter to rel subpackage.
[Kyle Weaver] [BEAM-12009] Reduce code duplication between rules by re-implementing
[Kyle Weaver] [BEAM-12009] Only match in BeamCalcSplittingRule if ≥ 1 of its component
[anup.d] BEAM-12166:Beam Sql - Combine Accumulator return Map fails with class
[suztomo] [BEAM-8357] Upgrading auto-value to 1.8 from 1.7.4
[Kenneth Knowles] Build source release zip from RC tag
[sychen] Add a BQ option for configuring buffering duration when auto-sharding is
[aromanenko.dev] [BEAM-12197] TPC-DS: Fix SQL-queries syntax
[suztomo] [BEAM-11010] Upgrading google-cloud-pubsublite to 0.13.2
[suztomo] [BEAM-11010] Copying SubscriberOptions from pubsublite repo
[suztomo] [BEAM-11010] Declaring flogger-system-backend to avoid conflicts
[Kyle Weaver] [BEAM-12194] Enable SqlTransform::registerUdaf in ZetaSQL.
[Kyle Weaver] [BEAM-12194] Code style changes from review.
[Boyuan Zhang] [BEAM-12114] Dataflow should apply KAFKA_READ_OVERRIDE when it's not
[noreply] Make sdk/worker_harness_container_image fully backwards compatible
[Kyle Weaver] [BEAM-12207] Remove log messages about files to stage.
[Kyle Weaver] [BEAM-12207] Enable nullness check on SparkPipelineRunner.java.
[noreply] [BEAM-12191] Add a test for python template generation with upload_graph
[noreply] [BEAM-7372] fix wrong usage of with_traceback (#14566)
[noreply] [BEAM-7372] cleanup codes for py2 from apache_beam/transforms (#14544)
[noreply] [BEAM-2085] Fixups for Python resource hints. (#14605)
------------------------------------------
[...truncated 3.81 MB...]
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-04-22_04_27_59-6126363272762420144
Apr 22, 2021 11:28:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:02.408Z: Autoscaling is enabled for job 2021-04-22_04_27_59-6126363272762420144. The number of workers will be between 1 and 1000.
Apr 22, 2021 11:28:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:02.710Z: Autoscaling was automatically enabled for job 2021-04-22_04_27_59-6126363272762420144.
Apr 22, 2021 11:28:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:09.081Z: Worker configuration: n1-standard-1 in us-central1-f.
Apr 22, 2021 11:28:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:09.829Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 22, 2021 11:28:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:09.957Z: Expanding GroupByKey operations into optimizable parts.
Apr 22, 2021 11:28:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:09.989Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 22, 2021 11:28:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.142Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 22, 2021 11:28:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.181Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
Apr 22, 2021 11:28:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.199Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.227Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.253Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.281Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.316Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.342Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.362Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.384Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.407Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.436Z: Unzipping flatten s12 for input s11.org.apache.beam.sdk.values.PCollection.<init>:402#5f2ef1f005ae0b4
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.461Z: Fusing unzipped copy of WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.490Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.516Z: Fusing consumer ParseGameEvent into TextIO.Read/Read
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.546Z: Fusing consumer ExtractUserScore/MapElements/Map into ParseGameEvent
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.571Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial into ExtractUserScore/MapElements/Map
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.617Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.652Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.677Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.700Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.730Z: Fusing consumer WriteUserScoreSums/ConvertToRow into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.755Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign into WriteUserScoreSums/ConvertToRow
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.781Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.828Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.861Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.885Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.911Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:10.935Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:11.252Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:11.339Z: Starting 1 workers in us-central1-f...
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:11.698Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 22, 2021 11:28:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:11.871Z: Executing operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 22, 2021 11:28:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:28:34.325Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 22, 2021 11:29:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:29:00.508Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 22, 2021 11:29:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:29:34.069Z: Workers have started successfully.
Apr 22, 2021 11:29:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:29:34.123Z: Workers have started successfully.
Apr 22, 2021 11:29:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-22T11:29:41.967Z: java.lang.NoClassDefFoundError: org/apache/beam/sdk/metrics/MetricsLogger
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.metrics.MetricsLogger
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 28 more
Apr 22, 2021 11:29:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-22T11:29:49.790Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 22, 2021 11:30:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-22T11:30:02.891Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 22, 2021 11:30:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-22T11:30:21.281Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 22, 2021 11:30:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:30:21.305Z: Finished operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 22, 2021 11:30:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-22T11:30:21.393Z: Workflow failed. Causes: S02:TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write failed., Internal Issue (ee0e1ccd559de9f3): 63963027:24514
Apr 22, 2021 11:30:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:30:21.452Z: Cleaning up.
Apr 22, 2021 11:30:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:30:21.533Z: Stopping worker pool...
Apr 22, 2021 11:31:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:31:12.003Z: Autoscaling: Resized worker pool from 1 to 0.
Apr 22, 2021 11:31:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-22T11:31:12.039Z: Worker pool stopped.
Apr 22, 2021 11:31:19 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-04-22_04_27_59-6126363272762420144 failed with status FAILED.
gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 30m 57s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/2yq7jmkwuybio
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1348
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1348/display/redirect?page=changes>
Changes:
[Robert Bradshaw] Populate runtime parameters in SDK worker startup.
[Robert Bradshaw] [BEAM-6597] Use runner capabilities to activate short ids.
[Robert Bradshaw] Fix wording.
[kileysok] Update Java beam container image
[Robert Bradshaw] Comment about runtime value usage.
[suztomo] Libraries-BOM 20.0.0 with gRPC and protobuf
[suztomo] Updating assertions for google-http-client 1.39.2
[suztomo] Declare gax-httpjson as a direct dependency of google-cloud-platform
[noreply] Add tensorboard-data-server license URL for Python container image.
[noreply] Update stringSlice to impl pkg.go.dev/flag#Getter.Get
[noreply] Add --sdk_container_image Python WorkerOption (#14557)
[noreply] [BEAM-2085] Add capability to declare resource hints in Python pipeline
[noreply] [BEAM-12174] Samza Portable Runner Support (#14554)
------------------------------------------
[...truncated 4.57 MB...]
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-04-21_04_28_04-15890605702330200531
Apr 21, 2021 11:28:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:08.126Z: Autoscaling is enabled for job 2021-04-21_04_28_04-15890605702330200531. The number of workers will be between 1 and 1000.
Apr 21, 2021 11:28:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:08.254Z: Autoscaling was automatically enabled for job 2021-04-21_04_28_04-15890605702330200531.
Apr 21, 2021 11:28:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:17.191Z: Worker configuration: n1-standard-1 in us-central1-a.
Apr 21, 2021 11:28:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:17.876Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 21, 2021 11:28:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.011Z: Expanding GroupByKey operations into optimizable parts.
Apr 21, 2021 11:28:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.042Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 21, 2021 11:28:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.239Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 21, 2021 11:28:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.263Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
Apr 21, 2021 11:28:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.295Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
Apr 21, 2021 11:28:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.326Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize into WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
Apr 21, 2021 11:28:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.351Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Finalize
Apr 21, 2021 11:28:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.385Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
Apr 21, 2021 11:28:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.416Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Apr 21, 2021 11:28:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.447Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Apr 21, 2021 11:28:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.469Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Apr 21, 2021 11:28:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.521Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Apr 21, 2021 11:28:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.548Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into WriteUserScoreSums/TextIO.Write/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.579Z: Unzipping flatten s12 for input s11.org.apache.beam.sdk.values.PCollection.<init>:402#5f2ef1f005ae0b4
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.606Z: Fusing unzipped copy of WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.622Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.660Z: Fusing consumer ParseGameEvent into TextIO.Read/Read
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.682Z: Fusing consumer ExtractUserScore/MapElements/Map into ParseGameEvent
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.704Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial into ExtractUserScore/MapElements/Map
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.729Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.750Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.773Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues into ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Read
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.788Z: Fusing consumer ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.809Z: Fusing consumer WriteUserScoreSums/ConvertToRow into ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Extract
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.832Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign into WriteUserScoreSums/ConvertToRow
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.856Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteUserScoreSums/TextIO.Write/WriteFiles/RewindowIntoGlobal/Window.Assign
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.878Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.910Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.944Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:18.979Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:19.006Z: Fusing consumer WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteUserScoreSums/TextIO.Write/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:19.369Z: Executing operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:19.446Z: Starting 1 workers in us-central1-a...
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:19.740Z: Finished operation ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Create
Apr 21, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:19.866Z: Executing operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 21, 2021 11:28:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:28:32.298Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 21, 2021 11:29:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:29:07.679Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 21, 2021 11:29:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:29:35.963Z: Workers have started successfully.
Apr 21, 2021 11:29:37 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:29:35.997Z: Workers have started successfully.
Apr 21, 2021 11:29:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-21T11:29:46.085Z: java.lang.NoClassDefFoundError: org/apache/beam/sdk/metrics/MetricsLogger
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.metrics.MetricsLogger
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 28 more
Apr 21, 2021 11:29:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-21T11:29:53.989Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 21, 2021 11:30:08 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-21T11:30:06.137Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 21, 2021 11:30:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-21T11:30:24.091Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 21, 2021 11:30:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:30:24.212Z: Finished operation TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write
Apr 21, 2021 11:30:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-21T11:30:24.312Z: Workflow failed. Causes: S02:TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write failed., Internal Issue (afd22392b3628ce4): 63963027:24514
Apr 21, 2021 11:30:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:30:24.385Z: Cleaning up.
Apr 21, 2021 11:30:26 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:30:24.464Z: Stopping worker pool...
Apr 21, 2021 11:31:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:31:17.690Z: Autoscaling: Resized worker pool from 1 to 0.
Apr 21, 2021 11:31:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-21T11:31:17.725Z: Worker pool stopped.
Apr 21, 2021 11:31:25 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-04-21_04_28_04-15890605702330200531 failed with status FAILED.
gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt*
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 31m 7s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/acnoqqwps63qc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostRelease_NightlySnapshot #1347
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1347/display/redirect?page=changes>
Changes:
[Elliotte Rusty Harold] use toMinutes
[Elliotte Rusty Harold] update test
[samuelw] [BEAM-12118] Fix racy precondition exception in QueueingBeamFnDataClient
[noreply] Change kafka table provider properties structure. (#14507)
[noreply] [BEAM-11839] Improve DeferredFrameTest._run_test (#14552)
[noreply] [BEAM-12191] Fix a bug that upload_graph doesn't reduce template file
[noreply] [BEAM-10029] Spanner IO read and write performance tests (#13857)
------------------------------------------
[...truncated 4.63 MB...]
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2021-04-20_04_28_07-11868786282907612053
Apr 20, 2021 11:28:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:10.763Z: Autoscaling is enabled for job 2021-04-20_04_28_07-11868786282907612053. The number of workers will be between 1 and 1000.
Apr 20, 2021 11:28:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:11.365Z: Autoscaling was automatically enabled for job 2021-04-20_04_28_07-11868786282907612053.
Apr 20, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:19.672Z: Worker configuration: n1-standard-1 in us-central1-c.
Apr 20, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:20.603Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 20, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:21.093Z: Expanding GroupByKey operations into optimizable parts.
Apr 20, 2021 11:28:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:21.209Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 20, 2021 11:28:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:21.783Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 20, 2021 11:28:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:21.895Z: Fusing consumer WriteCounts/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into WriteCounts/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)
Apr 20, 2021 11:28:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:22.009Z: Fusing consumer WriteCounts/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into WriteCounts/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
Apr 20, 2021 11:28:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:22.139Z: Fusing consumer WriteCounts/WriteFiles/FinalizeTempFileBundles/Finalize into WriteCounts/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
Apr 20, 2021 11:28:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:22.245Z: Fusing consumer WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into WriteCounts/WriteFiles/FinalizeTempFileBundles/Finalize
Apr 20, 2021 11:28:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:22.382Z: Fusing consumer WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
Apr 20, 2021 11:28:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:22.509Z: Fusing consumer WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Apr 20, 2021 11:28:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:22.638Z: Fusing consumer WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
Apr 20, 2021 11:28:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:22.747Z: Fusing consumer WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Apr 20, 2021 11:28:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:22.846Z: Fusing consumer WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Apr 20, 2021 11:28:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:22.955Z: Fusing consumer WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Apr 20, 2021 11:28:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:23.065Z: Unzipping flatten s12 for input s11.org.apache.beam.sdk.values.PCollection.<init>:402#5f2ef1f005ae0b4
Apr 20, 2021 11:28:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:23.196Z: Fusing unzipped copy of WriteCounts/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum
Apr 20, 2021 11:28:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:23.306Z: Fusing consumer WriteCounts/WriteFiles/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 20, 2021 11:28:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:23.413Z: Fusing consumer WordCount.CountWords/ParDo(ExtractWords) into ReadLines/Read
Apr 20, 2021 11:28:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:23.534Z: Fusing consumer WordCount.CountWords/Count.PerElement/Init/Map into WordCount.CountWords/ParDo(ExtractWords)
Apr 20, 2021 11:28:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:23.640Z: Fusing consumer WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues/Partial into WordCount.CountWords/Count.PerElement/Init/Map
Apr 20, 2021 11:28:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:23.755Z: Fusing consumer WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Reify into WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues/Partial
Apr 20, 2021 11:28:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:23.891Z: Fusing consumer WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Write into WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Reify
Apr 20, 2021 11:28:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:24.005Z: Fusing consumer WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues into WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Read
Apr 20, 2021 11:28:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:24.122Z: Fusing consumer WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues/Extract into WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues
Apr 20, 2021 11:28:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:24.232Z: Fusing consumer MapElements/Map into WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues/Extract
Apr 20, 2021 11:28:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:24.347Z: Fusing consumer WriteCounts/WriteFiles/RewindowIntoGlobal/Window.Assign into MapElements/Map
Apr 20, 2021 11:28:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:24.476Z: Fusing consumer WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into WriteCounts/WriteFiles/RewindowIntoGlobal/Window.Assign
Apr 20, 2021 11:28:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:24.589Z: Fusing consumer WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Apr 20, 2021 11:28:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:24.697Z: Fusing consumer WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
Apr 20, 2021 11:28:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:24.801Z: Fusing consumer WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
Apr 20, 2021 11:28:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:24.902Z: Fusing consumer WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten into WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
Apr 20, 2021 11:28:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:25.029Z: Fusing consumer WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum into WriteCounts/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten
Apr 20, 2021 11:28:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:26.260Z: Executing operation WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Create
Apr 20, 2021 11:28:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:26.542Z: Starting 1 workers in us-central1-c...
Apr 20, 2021 11:28:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:26.962Z: Finished operation WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Create
Apr 20, 2021 11:28:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:27.610Z: Executing operation ReadLines/Read+WordCount.CountWords/ParDo(ExtractWords)+WordCount.CountWords/Count.PerElement/Init/Map+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues/Partial+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Reify+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Write
Apr 20, 2021 11:28:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:28:39.213Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 20, 2021 11:29:19 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:29:18.544Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running stage(s).
Apr 20, 2021 11:29:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:29:54.175Z: Workers have started successfully.
Apr 20, 2021 11:29:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:29:54.286Z: Workers have started successfully.
Apr 20, 2021 11:30:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-20T11:30:02.564Z: java.lang.NoClassDefFoundError: org/apache/beam/sdk/metrics/MetricsLogger
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.metrics.MetricsLogger
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 28 more
Apr 20, 2021 11:30:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-20T11:30:07.324Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 20, 2021 11:30:22 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-20T11:30:20.527Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 20, 2021 11:30:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-20T11:30:30.058Z: java.lang.NoClassDefFoundError: org/apache/beam/runners/dataflow/worker/repackaged/org/apache/beam/runners/core/metrics/MetricsContainerImpl
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:106)
at org.apache.beam.runners.dataflow.worker.BatchModeExecutionContext$1.createContainer(BatchModeExecutionContext.java:103)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.lambda$getContainer$0(MetricsContainerRegistry.java:39)
at java.util.concurrent.ConcurrentSkipListMap.computeIfAbsent(ConcurrentSkipListMap.java:1682)
at org.apache.beam.runners.dataflow.worker.MetricsContainerRegistry.getContainer(MetricsContainerRegistry.java:39)
at org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.createOperationContext(DataflowExecutionContext.java:212)
at org.apache.beam.runners.dataflow.worker.SourceOperationExecutorFactory.create(SourceOperationExecutorFactory.java:49)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:376)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Apr 20, 2021 11:30:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:30:30.082Z: Finished operation ReadLines/Read+WordCount.CountWords/ParDo(ExtractWords)+WordCount.CountWords/Count.PerElement/Init/Map+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues/Partial+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Reify+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Write
Apr 20, 2021 11:30:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2021-04-20T11:30:30.397Z: Workflow failed. Causes: S02:ReadLines/Read+WordCount.CountWords/ParDo(ExtractWords)+WordCount.CountWords/Count.PerElement/Init/Map+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/Combine.GroupedValues/Partial+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Reify+WordCount.CountWords/Count.PerElement/Combine.perKey(Count)/GroupByKey/Write failed., Internal Issue (7213968c576c929c): 63963027:24514
Apr 20, 2021 11:30:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:30:30.656Z: Cleaning up.
Apr 20, 2021 11:30:31 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:30:30.911Z: Stopping worker pool...
Apr 20, 2021 11:31:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:31:20.760Z: Autoscaling: Resized worker pool from 1 to 0.
Apr 20, 2021 11:31:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-04-20T11:31:20.895Z: Worker pool stopped.
Apr 20, 2021 11:31:31 AM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2021-04-20_04_28_07-11868786282907612053 failed with status FAILED.
gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/count* | grep Montague:
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/count*
CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/count*
[ERROR] Failed command
> Task :runners:google-cloud-dataflow-java:runQuickstartJavaDataflow FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:google-cloud-dataflow-java:runQuickstartJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 31m 12s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/wd33r6tymwjcw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org