You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2023/04/11 01:49:39 UTC
Build failed in Jenkins: beam_PerformanceTests_Kafka_IO #3727
See <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/3727/display/redirect?page=changes>
Changes:
[noreply] Touch a dummy file for go licenses. (#26183)
------------------------------------------
[...truncated 988.18 KB...]
INFO: Adding Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues as step s14
Apr 11, 2023 1:19:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Values/Values/Map as step s15
Apr 11, 2023 1:19:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) as step s16
Apr 11, 2023 1:19:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) as step s17
Apr 11, 2023 1:19:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Create.Values/Read(CreateSource) as step s18
Apr 11, 2023 1:19:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Flatten.PCollections as step s19
Apr 11, 2023 1:19:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Window.Into()/Flatten.PCollections as step s20
Apr 11, 2023 1:19:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/WithKeys/AddKeys/Map as step s21
Apr 11, 2023 1:19:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/GroupByKey as step s22
Apr 11, 2023 1:19:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Values/Values/Map as step s23
Apr 11, 2023 1:19:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(Concat) as step s24
Apr 11, 2023 1:19:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GetPane/Map as step s25
Apr 11, 2023 1:19:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/RunChecks as step s26
Apr 11, 2023 1:19:34 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/VerifyAssertions/ParDo(DefaultConclude) as step s27
Apr 11, 2023 1:19:34 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.48.0-SNAPSHOT
Apr 11, 2023 1:19:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-04-10_18_19_34-3383914971386477877?project=apache-beam-testing
Apr 11, 2023 1:19:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2023-04-10_18_19_34-3383914971386477877
Apr 11, 2023 1:19:35 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2023-04-10_18_19_34-3383914971386477877
Apr 11, 2023 1:19:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2023-04-11T01:19:37.995Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: kafkaioit0testkafkaioreadsandwritescorrectlyinbatch-jenkin-owcx. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Apr 11, 2023 1:19:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:47.883Z: Worker configuration: e2-standard-2 in us-central1-b.
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.119Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.265Z: Expanding GroupByKey operations into optimizable parts.
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.296Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.441Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.475Z: Elided trivial flatten
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.508Z: Unzipping flatten s19 for input s17.org.apache.beam.sdk.values.PCollection.<init>:405#e63ad18dfc139ac0
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.529Z: Fusing unzipped copy of PAssert$0/GroupGlobally/WithKeys/AddKeys/Map, through flatten PAssert$0/GroupGlobally/Flatten.PCollections, into producer PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.551Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/Write into PAssert$0/GroupGlobally/GroupByKey/Reify
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.576Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/GroupByWindow into PAssert$0/GroupGlobally/GroupByKey/Read
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.598Z: Fusing consumer PAssert$0/GroupGlobally/Values/Values/Map into PAssert$0/GroupGlobally/GroupByKey/GroupByWindow
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.622Z: Fusing consumer PAssert$0/GroupGlobally/ParDo(Concat) into PAssert$0/GroupGlobally/Values/Values/Map
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.645Z: Fusing consumer PAssert$0/GetPane/Map into PAssert$0/GroupGlobally/ParDo(Concat)
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.665Z: Fusing consumer PAssert$0/RunChecks into PAssert$0/GetPane/Map
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.697Z: Fusing consumer PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.719Z: Unzipping flatten s19-u40 for input s21.org.apache.beam.sdk.values.PCollection.<init>:405#24aa17775bf242ac-c38
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.751Z: Fusing unzipped copy of PAssert$0/GroupGlobally/GroupByKey/Reify, through flatten PAssert$0/GroupGlobally/Flatten.PCollections/Unzipped-1, into producer PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.781Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Create/Read(CreateSource)
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.808Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.834Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.856Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.889Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.920Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Read
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.954Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:49.978Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:50.008Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Read into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:50.042Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/StripIds into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Read
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:50.065Z: Fusing consumer Measure read time into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/StripIds
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:50.098Z: Fusing consumer Map records to strings/Map into Measure read time
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:50.122Z: Fusing consumer Calculate hashcode/WithKeys/AddKeys/Map into Map records to strings/Map
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:50.145Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate hashcode/WithKeys/AddKeys/Map
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:50.175Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:50.197Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:50.229Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:50.260Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:50.286Z: Fusing consumer Calculate hashcode/Values/Values/Map into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:50.308Z: Fusing consumer PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) into Calculate hashcode/Values/Values/Map
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:50.336Z: Fusing consumer PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) into PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:50.368Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/Reify into PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:50.403Z: Fusing consumer PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:50.755Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Create
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:50.822Z: Starting 5 ****s in us-central1-b...
Apr 11, 2023 1:19:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:51.127Z: Finished operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Create
Apr 11, 2023 1:19:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:19:51.257Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Create/Read(CreateSource)+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write
Apr 11, 2023 1:20:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:20:12.139Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 11, 2023 1:20:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:20:36.572Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
Apr 11, 2023 1:21:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:21:06.500Z: Workers have started successfully.
Apr 11, 2023 1:21:30 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:21:28.622Z: All ****s have finished the startup processes and began to receive work requests.
Apr 11, 2023 1:21:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:21:33.352Z: Finished operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Create/Read(CreateSource)+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write
Apr 11, 2023 1:21:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:21:33.411Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Close
Apr 11, 2023 1:21:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:21:33.670Z: Finished operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Close
Apr 11, 2023 1:21:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:21:33.731Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Apr 11, 2023 1:21:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:21:33.893Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Apr 11, 2023 1:21:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-11T01:21:34.019Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Read+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Read+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/StripIds+Measure read time+Map records to strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Apr 11, 2023 1:49:35 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned within allotted timeout. State value RUNNING
Apr 11, 2023 1:49:35 AM org.apache.kafka.common.config.AbstractConfig logAll
INFO: AdminClientConfig values:
bootstrap.servers = [34.68.52.184:32401, 34.134.200.129:32402, 104.198.237.27:32403]
client.dns.lookup = default
client.id =
connections.max.idle.ms = 300000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 120000
retries = 5
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
security.providers = null
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
Apr 11, 2023 1:49:35 AM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka version: 2.4.1
Apr 11, 2023 1:49:35 AM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka commitId: c57222ae8cd7866b
Apr 11, 2023 1:49:35 AM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka startTimeMs: 1681177775897
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:kafka:integrationTest FAILED
org.apache.beam.sdk.io.kafka.KafkaIOIT > testKafkaIOReadsAndWritesCorrectlyInBatch FAILED
java.lang.AssertionError: expected:<DONE> but was:<null>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:120)
at org.junit.Assert.assertEquals(Assert.java:146)
at org.apache.beam.sdk.io.kafka.KafkaIOIT.testKafkaIOReadsAndWritesCorrectlyInBatch(KafkaIOIT.java:275)
1 test completed, 1 failed
Finished generating test XML results (0.026 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.045 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/reports/tests/integrationTest>
:sdks:java:io:kafka:integrationTest (Thread[Execution **** Thread 2,5,main]) completed. Took 38 mins 0.153 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:kafka:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/reports/tests/integrationTest/index.html>
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 38m 22s
143 actionable tasks: 1 executed, 142 up-to-date
Publishing build scan...
https://gradle.com/s/kdbclcr7ohhe4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PerformanceTests_Kafka_IO #3734
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/3734/display/redirect>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_Kafka_IO #3733
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/3733/display/redirect>
Changes:
------------------------------------------
[...truncated 994.05 KB...]
INFO: Adding PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) as step s16
Apr 14, 2023 1:16:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) as step s17
Apr 14, 2023 1:16:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Create.Values/Read(CreateSource) as step s18
Apr 14, 2023 1:16:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Flatten.PCollections as step s19
Apr 14, 2023 1:16:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Window.Into()/Flatten.PCollections as step s20
Apr 14, 2023 1:16:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/WithKeys/AddKeys/Map as step s21
Apr 14, 2023 1:16:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/GroupByKey as step s22
Apr 14, 2023 1:16:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Values/Values/Map as step s23
Apr 14, 2023 1:16:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(Concat) as step s24
Apr 14, 2023 1:16:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GetPane/Map as step s25
Apr 14, 2023 1:16:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/RunChecks as step s26
Apr 14, 2023 1:16:41 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/VerifyAssertions/ParDo(DefaultConclude) as step s27
Apr 14, 2023 1:16:41 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.48.0-SNAPSHOT
Apr 14, 2023 1:16:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-04-13_18_16_41-5651989897113113401?project=apache-beam-testing
Apr 14, 2023 1:16:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2023-04-13_18_16_41-5651989897113113401
Apr 14, 2023 1:16:42 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2023-04-13_18_16_41-5651989897113113401
Apr 14, 2023 1:16:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2023-04-14T01:16:45.252Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: kafkaioit0testkafkaioreadsandwritescorrectlyinbatch-jenkin-zrc3. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:56.731Z: Worker configuration: e2-standard-2 in us-central1-b.
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:57.642Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:57.826Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:57.951Z: Expanding GroupByKey operations into optimizable parts.
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:57.988Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.156Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.182Z: Elided trivial flatten
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.211Z: Unzipping flatten s19 for input s17.org.apache.beam.sdk.values.PCollection.<init>:405#e63ad18dfc139ac0
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.248Z: Fusing unzipped copy of PAssert$0/GroupGlobally/WithKeys/AddKeys/Map, through flatten PAssert$0/GroupGlobally/Flatten.PCollections, into producer PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.277Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/Write into PAssert$0/GroupGlobally/GroupByKey/Reify
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.313Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/GroupByWindow into PAssert$0/GroupGlobally/GroupByKey/Read
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.338Z: Fusing consumer PAssert$0/GroupGlobally/Values/Values/Map into PAssert$0/GroupGlobally/GroupByKey/GroupByWindow
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.366Z: Fusing consumer PAssert$0/GroupGlobally/ParDo(Concat) into PAssert$0/GroupGlobally/Values/Values/Map
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.399Z: Fusing consumer PAssert$0/GetPane/Map into PAssert$0/GroupGlobally/ParDo(Concat)
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.431Z: Fusing consumer PAssert$0/RunChecks into PAssert$0/GetPane/Map
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.463Z: Fusing consumer PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.497Z: Unzipping flatten s19-u40 for input s21.org.apache.beam.sdk.values.PCollection.<init>:405#24aa17775bf242ac-c38
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.521Z: Fusing unzipped copy of PAssert$0/GroupGlobally/GroupByKey/Reify, through flatten PAssert$0/GroupGlobally/Flatten.PCollections/Unzipped-1, into producer PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.541Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Create/Read(CreateSource)
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.562Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.586Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.621Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.643Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.677Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Read
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.698Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.725Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.760Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Read into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.793Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/StripIds into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Read
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.815Z: Fusing consumer Measure read time into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/StripIds
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.847Z: Fusing consumer Map records to strings/Map into Measure read time
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.873Z: Fusing consumer Calculate hashcode/WithKeys/AddKeys/Map into Map records to strings/Map
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.911Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate hashcode/WithKeys/AddKeys/Map
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.936Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.962Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:58.993Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:59.020Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:59.049Z: Fusing consumer Calculate hashcode/Values/Values/Map into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:59.075Z: Fusing consumer PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) into Calculate hashcode/Values/Values/Map
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:59.109Z: Fusing consumer PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) into PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:59.143Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/Reify into PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Apr 14, 2023 1:16:59 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:59.175Z: Fusing consumer PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Apr 14, 2023 1:17:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:59.520Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Create
Apr 14, 2023 1:17:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:59.598Z: Starting 5 ****s in us-central1-b...
Apr 14, 2023 1:17:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:16:59.844Z: Finished operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Create
Apr 14, 2023 1:17:00 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:17:00.000Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Create/Read(CreateSource)+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write
Apr 14, 2023 1:17:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:17:53.051Z: Autoscaling: Raised the number of ****s to 3 based on the rate of progress in the currently running stage(s).
Apr 14, 2023 1:17:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:17:53.084Z: Autoscaling: Resized **** pool to 3, though goal was 5. This could be a quota issue.
Apr 14, 2023 1:18:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:18:02.874Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
Apr 14, 2023 1:18:28 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:18:27.934Z: Workers have started successfully.
Apr 14, 2023 1:18:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:18:47.626Z: All ****s have finished the startup processes and began to receive work requests.
Apr 14, 2023 1:18:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:18:50.413Z: Finished operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Create/Read(CreateSource)+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write
Apr 14, 2023 1:18:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:18:50.469Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Close
Apr 14, 2023 1:18:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:18:50.541Z: Finished operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Close
Apr 14, 2023 1:18:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:18:50.611Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Apr 14, 2023 1:18:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:18:50.745Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Apr 14, 2023 1:18:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-14T01:18:50.888Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Read+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Read+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/StripIds+Measure read time+Map records to strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Apr 14, 2023 1:46:42 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned within allotted timeout. State value RUNNING
Apr 14, 2023 1:46:42 AM org.apache.kafka.common.config.AbstractConfig logAll
INFO: AdminClientConfig values:
bootstrap.servers = [34.170.207.179:32401, 35.238.93.205:32402, 34.27.161.95:32403]
client.dns.lookup = default
client.id =
connections.max.idle.ms = 300000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 120000
retries = 5
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
security.providers = null
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
Apr 14, 2023 1:46:42 AM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka version: 2.4.1
Apr 14, 2023 1:46:42 AM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka commitId: c57222ae8cd7866b
Apr 14, 2023 1:46:42 AM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka startTimeMs: 1681436802899
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:kafka:integrationTest FAILED
org.apache.beam.sdk.io.kafka.KafkaIOIT > testKafkaIOReadsAndWritesCorrectlyInBatch FAILED
java.lang.AssertionError: expected:<DONE> but was:<null>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:120)
at org.junit.Assert.assertEquals(Assert.java:146)
at org.apache.beam.sdk.io.kafka.KafkaIOIT.testKafkaIOReadsAndWritesCorrectlyInBatch(KafkaIOIT.java:275)
1 test completed, 1 failed
Finished generating test XML results (0.027 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.087 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/reports/tests/integrationTest>
:sdks:java:io:kafka:integrationTest (Thread[Execution ****,5,main]) completed. Took 37 mins 20.363 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:kafka:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/reports/tests/integrationTest/index.html>
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 37m 47s
143 actionable tasks: 1 executed, 142 up-to-date
Publishing build scan...
https://gradle.com/s/io6vo66xdorae
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_Kafka_IO #3732
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/3732/display/redirect>
Changes:
------------------------------------------
[...truncated 989.15 KB...]
INFO: Adding Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues as step s14
Apr 13, 2023 1:12:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Values/Values/Map as step s15
Apr 13, 2023 1:12:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) as step s16
Apr 13, 2023 1:12:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) as step s17
Apr 13, 2023 1:12:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Create.Values/Read(CreateSource) as step s18
Apr 13, 2023 1:12:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Flatten.PCollections as step s19
Apr 13, 2023 1:12:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Window.Into()/Flatten.PCollections as step s20
Apr 13, 2023 1:12:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/WithKeys/AddKeys/Map as step s21
Apr 13, 2023 1:12:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/GroupByKey as step s22
Apr 13, 2023 1:12:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Values/Values/Map as step s23
Apr 13, 2023 1:12:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(Concat) as step s24
Apr 13, 2023 1:12:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GetPane/Map as step s25
Apr 13, 2023 1:12:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/RunChecks as step s26
Apr 13, 2023 1:12:23 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/VerifyAssertions/ParDo(DefaultConclude) as step s27
Apr 13, 2023 1:12:23 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.48.0-SNAPSHOT
Apr 13, 2023 1:12:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-04-13_06_12_23-1531355615906922422?project=apache-beam-testing
Apr 13, 2023 1:12:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2023-04-13_06_12_23-1531355615906922422
Apr 13, 2023 1:12:24 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2023-04-13_06_12_23-1531355615906922422
Apr 13, 2023 1:12:28 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2023-04-13T13:12:26.350Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: kafkaioit0testkafkaioreadsandwritescorrectlyinbatch-jenkin-obqz. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Apr 13, 2023 1:12:37 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:35.992Z: Worker configuration: e2-standard-2 in us-central1-b.
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.065Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.202Z: Expanding GroupByKey operations into optimizable parts.
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.226Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.353Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.381Z: Elided trivial flatten
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.406Z: Unzipping flatten s19 for input s17.org.apache.beam.sdk.values.PCollection.<init>:405#e63ad18dfc139ac0
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.437Z: Fusing unzipped copy of PAssert$0/GroupGlobally/WithKeys/AddKeys/Map, through flatten PAssert$0/GroupGlobally/Flatten.PCollections, into producer PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.469Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/Write into PAssert$0/GroupGlobally/GroupByKey/Reify
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.492Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/GroupByWindow into PAssert$0/GroupGlobally/GroupByKey/Read
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.525Z: Fusing consumer PAssert$0/GroupGlobally/Values/Values/Map into PAssert$0/GroupGlobally/GroupByKey/GroupByWindow
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.546Z: Fusing consumer PAssert$0/GroupGlobally/ParDo(Concat) into PAssert$0/GroupGlobally/Values/Values/Map
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.567Z: Fusing consumer PAssert$0/GetPane/Map into PAssert$0/GroupGlobally/ParDo(Concat)
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.596Z: Fusing consumer PAssert$0/RunChecks into PAssert$0/GetPane/Map
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.619Z: Fusing consumer PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.651Z: Unzipping flatten s19-u40 for input s21.org.apache.beam.sdk.values.PCollection.<init>:405#24aa17775bf242ac-c38
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.684Z: Fusing unzipped copy of PAssert$0/GroupGlobally/GroupByKey/Reify, through flatten PAssert$0/GroupGlobally/Flatten.PCollections/Unzipped-1, into producer PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.714Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Create/Read(CreateSource)
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.747Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.780Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.811Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.844Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.865Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Read
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.896Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.919Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.950Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Read into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:37.971Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/StripIds into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Read
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:38.002Z: Fusing consumer Measure read time into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/StripIds
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:38.035Z: Fusing consumer Map records to strings/Map into Measure read time
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:38.061Z: Fusing consumer Calculate hashcode/WithKeys/AddKeys/Map into Map records to strings/Map
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:38.087Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate hashcode/WithKeys/AddKeys/Map
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:38.112Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:38.135Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:38.159Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:38.186Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:38.208Z: Fusing consumer Calculate hashcode/Values/Values/Map into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:38.231Z: Fusing consumer PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) into Calculate hashcode/Values/Values/Map
Apr 13, 2023 1:12:38 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:38.255Z: Fusing consumer PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) into PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)
Apr 13, 2023 1:12:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:38.294Z: Fusing consumer PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Apr 13, 2023 1:12:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:38.315Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/Reify into PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Apr 13, 2023 1:12:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:38.636Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Create
Apr 13, 2023 1:12:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:38.706Z: Starting 5 ****s in us-central1-b...
Apr 13, 2023 1:12:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:38.959Z: Finished operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Create
Apr 13, 2023 1:12:40 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:39.071Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Create/Read(CreateSource)+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write
Apr 13, 2023 1:12:59 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:12:58.299Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 13, 2023 1:13:23 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:13:22.223Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
Apr 13, 2023 1:13:52 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:13:51.514Z: Workers have started successfully.
Apr 13, 2023 1:14:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:14:10.561Z: All ****s have finished the startup processes and began to receive work requests.
Apr 13, 2023 1:14:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:14:12.811Z: Finished operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Create/Read(CreateSource)+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write
Apr 13, 2023 1:14:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:14:12.861Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Close
Apr 13, 2023 1:14:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:14:14.299Z: Finished operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Close
Apr 13, 2023 1:14:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:14:14.364Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Apr 13, 2023 1:14:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:14:14.480Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Apr 13, 2023 1:14:15 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T13:14:14.607Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Read+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Read+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/StripIds+Measure read time+Map records to strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Apr 13, 2023 1:42:24 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned within allotted timeout. State value RUNNING
Apr 13, 2023 1:42:24 PM org.apache.kafka.common.config.AbstractConfig logAll
INFO: AdminClientConfig values:
bootstrap.servers = [34.171.33.207:32404, 34.68.126.75:32405, 34.27.211.164:32406]
client.dns.lookup = default
client.id =
connections.max.idle.ms = 300000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 120000
retries = 5
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
security.providers = null
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
Apr 13, 2023 1:42:24 PM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka version: 2.4.1
Apr 13, 2023 1:42:24 PM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka commitId: c57222ae8cd7866b
Apr 13, 2023 1:42:24 PM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka startTimeMs: 1681393344266
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:kafka:integrationTest FAILED
org.apache.beam.sdk.io.kafka.KafkaIOIT > testKafkaIOReadsAndWritesCorrectlyInBatch FAILED
java.lang.AssertionError: expected:<DONE> but was:<null>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:120)
at org.junit.Assert.assertEquals(Assert.java:146)
at org.apache.beam.sdk.io.kafka.KafkaIOIT.testKafkaIOReadsAndWritesCorrectlyInBatch(KafkaIOIT.java:275)
1 test completed, 1 failed
Finished generating test XML results (0.026 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.043 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/reports/tests/integrationTest>
:sdks:java:io:kafka:integrationTest (Thread[Execution **** Thread 5,5,main]) completed. Took 37 mins 6.849 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:kafka:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/reports/tests/integrationTest/index.html>
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 37m 30s
143 actionable tasks: 1 executed, 142 up-to-date
Publishing build scan...
https://gradle.com/s/mqiy4y6epx34y
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_Kafka_IO #3731
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/3731/display/redirect?page=changes>
Changes:
[noreply] Add manual trigger to stale (#26248)
------------------------------------------
[...truncated 986.80 KB...]
INFO: Adding Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues as step s14
Apr 13, 2023 1:15:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Values/Values/Map as step s15
Apr 13, 2023 1:15:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) as step s16
Apr 13, 2023 1:15:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) as step s17
Apr 13, 2023 1:15:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Create.Values/Read(CreateSource) as step s18
Apr 13, 2023 1:15:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Flatten.PCollections as step s19
Apr 13, 2023 1:15:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Window.Into()/Flatten.PCollections as step s20
Apr 13, 2023 1:15:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/WithKeys/AddKeys/Map as step s21
Apr 13, 2023 1:15:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/GroupByKey as step s22
Apr 13, 2023 1:15:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Values/Values/Map as step s23
Apr 13, 2023 1:15:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(Concat) as step s24
Apr 13, 2023 1:15:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GetPane/Map as step s25
Apr 13, 2023 1:15:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/RunChecks as step s26
Apr 13, 2023 1:15:24 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/VerifyAssertions/ParDo(DefaultConclude) as step s27
Apr 13, 2023 1:15:24 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.48.0-SNAPSHOT
Apr 13, 2023 1:15:24 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-04-12_18_15_24-7005488136515558269?project=apache-beam-testing
Apr 13, 2023 1:15:24 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2023-04-12_18_15_24-7005488136515558269
Apr 13, 2023 1:15:24 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2023-04-12_18_15_24-7005488136515558269
Apr 13, 2023 1:15:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2023-04-13T01:15:27.367Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: kafkaioit0testkafkaioreadsandwritescorrectlyinbatch-jenkin-y9ql. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Apr 13, 2023 1:15:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:38.152Z: Worker configuration: e2-standard-2 in us-central1-b.
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:39.358Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:39.492Z: Expanding GroupByKey operations into optimizable parts.
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:39.528Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:39.679Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:39.712Z: Elided trivial flatten
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:39.735Z: Unzipping flatten s19 for input s17.org.apache.beam.sdk.values.PCollection.<init>:405#e63ad18dfc139ac0
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:39.763Z: Fusing unzipped copy of PAssert$0/GroupGlobally/WithKeys/AddKeys/Map, through flatten PAssert$0/GroupGlobally/Flatten.PCollections, into producer PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:39.795Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/Write into PAssert$0/GroupGlobally/GroupByKey/Reify
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:39.829Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/GroupByWindow into PAssert$0/GroupGlobally/GroupByKey/Read
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:39.850Z: Fusing consumer PAssert$0/GroupGlobally/Values/Values/Map into PAssert$0/GroupGlobally/GroupByKey/GroupByWindow
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:39.875Z: Fusing consumer PAssert$0/GroupGlobally/ParDo(Concat) into PAssert$0/GroupGlobally/Values/Values/Map
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:39.903Z: Fusing consumer PAssert$0/GetPane/Map into PAssert$0/GroupGlobally/ParDo(Concat)
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:39.929Z: Fusing consumer PAssert$0/RunChecks into PAssert$0/GetPane/Map
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:39.949Z: Fusing consumer PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:39.973Z: Unzipping flatten s19-u40 for input s21.org.apache.beam.sdk.values.PCollection.<init>:405#24aa17775bf242ac-c38
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.000Z: Fusing unzipped copy of PAssert$0/GroupGlobally/GroupByKey/Reify, through flatten PAssert$0/GroupGlobally/Flatten.PCollections/Unzipped-1, into producer PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.032Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Create/Read(CreateSource)
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.055Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.077Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.101Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign
Apr 13, 2023 1:15:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.138Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.176Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Read
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.206Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.239Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.262Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Read into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.291Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/StripIds into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Read
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.323Z: Fusing consumer Measure read time into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/StripIds
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.346Z: Fusing consumer Map records to strings/Map into Measure read time
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.381Z: Fusing consumer Calculate hashcode/WithKeys/AddKeys/Map into Map records to strings/Map
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.413Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate hashcode/WithKeys/AddKeys/Map
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.436Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.464Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.494Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.519Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.544Z: Fusing consumer Calculate hashcode/Values/Values/Map into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.572Z: Fusing consumer PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) into Calculate hashcode/Values/Values/Map
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.597Z: Fusing consumer PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) into PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.626Z: Fusing consumer PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:40.652Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/Reify into PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:41.008Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Create
Apr 13, 2023 1:15:41 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:41.085Z: Starting 5 ****s in us-central1-b...
Apr 13, 2023 1:15:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:41.515Z: Finished operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Create
Apr 13, 2023 1:15:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:41.671Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Create/Read(CreateSource)+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write
Apr 13, 2023 1:15:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:15:46.864Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 13, 2023 1:16:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:16:19.370Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
Apr 13, 2023 1:16:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:16:56.102Z: Workers have started successfully.
Apr 13, 2023 1:17:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:17:20.700Z: All ****s have finished the startup processes and began to receive work requests.
Apr 13, 2023 1:17:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:17:24.224Z: Finished operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Create/Read(CreateSource)+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write
Apr 13, 2023 1:17:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:17:24.294Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Close
Apr 13, 2023 1:17:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:17:25.293Z: Finished operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Close
Apr 13, 2023 1:17:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:17:25.418Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Apr 13, 2023 1:17:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:17:25.560Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Apr 13, 2023 1:17:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-13T01:17:25.717Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Read+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Read+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/StripIds+Measure read time+Map records to strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Apr 13, 2023 1:45:24 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned within allotted timeout. State value RUNNING
Apr 13, 2023 1:45:25 AM org.apache.kafka.common.config.AbstractConfig logAll
INFO: AdminClientConfig values:
bootstrap.servers = [34.30.47.84:32401, 34.29.20.201:32402, 34.31.82.88:32403]
client.dns.lookup = default
client.id =
connections.max.idle.ms = 300000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 120000
retries = 5
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
security.providers = null
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
Apr 13, 2023 1:45:25 AM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka version: 2.4.1
Apr 13, 2023 1:45:25 AM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka commitId: c57222ae8cd7866b
Apr 13, 2023 1:45:25 AM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka startTimeMs: 1681350325130
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:kafka:integrationTest FAILED
org.apache.beam.sdk.io.kafka.KafkaIOIT > testKafkaIOReadsAndWritesCorrectlyInBatch FAILED
java.lang.AssertionError: expected:<DONE> but was:<null>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:120)
at org.junit.Assert.assertEquals(Assert.java:146)
at org.apache.beam.sdk.io.kafka.KafkaIOIT.testKafkaIOReadsAndWritesCorrectlyInBatch(KafkaIOIT.java:275)
1 test completed, 1 failed
Finished generating test XML results (0.024 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.038 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/reports/tests/integrationTest>
:sdks:java:io:kafka:integrationTest (Thread[Execution **** Thread 2,5,main]) completed. Took 37 mins 5.268 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:kafka:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/reports/tests/integrationTest/index.html>
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 37m 26s
143 actionable tasks: 1 executed, 142 up-to-date
Publishing build scan...
https://gradle.com/s/wrsfeucqx4e3i
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_Kafka_IO #3730
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/3730/display/redirect>
Changes:
------------------------------------------
[...truncated 616.36 KB...]
request.timeout.ms = 30000
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
security.providers = null
send.buffer.bytes = 131072
session.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
Apr 12, 2023 12:45:44 PM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka version: 2.4.1
Apr 12, 2023 12:45:44 PM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka commitId: c57222ae8cd7866b
Apr 12, 2023 12:45:44 PM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka startTimeMs: 1681303544038
Apr 12, 2023 12:45:44 PM org.apache.kafka.clients.Metadata update
INFO: [Consumer clientId=consumer-1, groupId=null] Cluster ID: _xQqEA7kSZqjE70ELAWqsw
Apr 12, 2023 12:45:44 PM org.apache.beam.sdk.io.kafka.KafkaUnboundedSource split
INFO: Partitions assigned to split 0 (total 1): beam-sdf-0
Apr 12, 2023 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Read from unbounded Kafka/KafkaIO.Read.ReadFromKafkaViaSDF/Read(KafkaUnboundedSource)/StripIds as step s2
Apr 12, 2023 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Measure read time as step s3
Apr 12, 2023 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Map records to strings/Map as step s4
Apr 12, 2023 12:45:44 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Counting element as step s5
Apr 12, 2023 12:45:44 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.48.0-SNAPSHOT
Apr 12, 2023 12:45:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-04-12_05_45_44-9026980847874937419?project=apache-beam-testing
Apr 12, 2023 12:45:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2023-04-12_05_45_44-9026980847874937419
Apr 12, 2023 12:45:45 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2023-04-12_05_45_44-9026980847874937419
Apr 12, 2023 12:45:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2023-04-12T12:45:48.877Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: kafkaioit0testkafkaioreadsandwritescorrectlyinstreaming-je-45w0. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Apr 12, 2023 12:45:58 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:45:58.154Z: Worker configuration: e2-standard-2 in us-central1-f.
Apr 12, 2023 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:45:59.444Z: Expanding SplittableParDo operations into optimizable parts.
Apr 12, 2023 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:45:59.470Z: Expanding CollectionToSingleton operations into optimizable parts.
Apr 12, 2023 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:45:59.521Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 12, 2023 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:45:59.570Z: Expanding SplittableProcessKeyed operations into optimizable parts.
Apr 12, 2023 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:45:59.597Z: Expanding GroupByKey operations into streaming Read/Write steps
Apr 12, 2023 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:45:59.617Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 12, 2023 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:45:59.686Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 12, 2023 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:45:59.714Z: Fusing consumer Read from unbounded Kafka/KafkaIO.Read.ReadFromKafkaViaSDF/ParDo(GenerateKafkaSourceDescriptor)/ParMultiDo(GenerateKafkaSourceDescriptor) into Read from unbounded Kafka/KafkaIO.Read.ReadFromKafkaViaSDF/Impulse
Apr 12, 2023 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:45:59.747Z: Fusing consumer Read-from-unbounded-Kafka-KafkaIO-Read-ReadFromKafkaViaSDF-KafkaIO-ReadSourceDescriptors-ParDo-Unbou/PairWithRestriction into Read from unbounded Kafka/KafkaIO.Read.ReadFromKafkaViaSDF/ParDo(GenerateKafkaSourceDescriptor)/ParMultiDo(GenerateKafkaSourceDescriptor)
Apr 12, 2023 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:45:59.769Z: Fusing consumer Read-from-unbounded-Kafka-KafkaIO-Read-ReadFromKafkaViaSDF-KafkaIO-ReadSourceDescriptors-ParDo-Unbou/SplitWithSizing into Read-from-unbounded-Kafka-KafkaIO-Read-ReadFromKafkaViaSDF-KafkaIO-ReadSourceDescriptors-ParDo-Unbou/PairWithRestriction
Apr 12, 2023 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:45:59.795Z: Fusing consumer Read from unbounded Kafka/KafkaIO.Read.ReadFromKafkaViaSDF/KafkaIO.ReadSourceDescriptors/MapElements/Map/ParMultiDo(Anonymous) into Read-from-unbounded-Kafka-KafkaIO-Read-ReadFromKafkaViaSDF-KafkaIO-ReadSourceDescriptors-ParDo-Unbou/ProcessElementAndRestrictionWithSizing
Apr 12, 2023 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:45:59.818Z: Fusing consumer Measure read time/ParMultiDo(TimeMonitor) into Read from unbounded Kafka/KafkaIO.Read.ReadFromKafkaViaSDF/KafkaIO.ReadSourceDescriptors/MapElements/Map/ParMultiDo(Anonymous)
Apr 12, 2023 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:45:59.844Z: Fusing consumer Map records to strings/Map/ParMultiDo(Anonymous) into Measure read time/ParMultiDo(TimeMonitor)
Apr 12, 2023 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:45:59.870Z: Fusing consumer Counting element/ParMultiDo(Counting) into Map records to strings/Map/ParMultiDo(Anonymous)
Apr 12, 2023 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:45:59.973Z: Running job using Streaming Engine
Apr 12, 2023 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:46:00.375Z: Starting 5 ****s in us-central1-f...
Apr 12, 2023 12:46:01 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:46:00.725Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 12, 2023 12:46:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:46:43.477Z: Autoscaling: Raised the number of ****s to 5 so that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 12, 2023 12:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:47:33.950Z: Workers have started successfully.
Apr 12, 2023 12:47:35 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T12:47:34.262Z: All ****s have finished the startup processes and began to receive work requests.
Apr 12, 2023 1:00:33 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2023-04-12T13:00:32.880Z: The ****s of given job are going to be updated.
Apr 12, 2023 1:10:45 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned within allotted timeout. State value RUNNING
Apr 12, 2023 1:10:45 PM org.apache.kafka.common.config.AbstractConfig logAll
INFO: AdminClientConfig values:
bootstrap.servers = [34.27.196.147:32404, 34.66.246.7:32405, 35.184.46.216:32406]
client.dns.lookup = default
client.id =
connections.max.idle.ms = 300000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 120000
retries = 5
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
security.providers = null
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
Apr 12, 2023 1:10:45 PM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka version: 2.4.1
Apr 12, 2023 1:10:45 PM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka commitId: c57222ae8cd7866b
Apr 12, 2023 1:10:45 PM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka startTimeMs: 1681305045134
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:kafka:integrationTest FAILED
org.apache.beam.sdk.io.kafka.KafkaIOIT > testKafkaIOReadsAndWritesCorrectlyInStreaming FAILED
java.lang.AssertionError: actual number of records 99381953 smaller than expected: 100000000.
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.apache.beam.sdk.io.kafka.KafkaIOIT.testKafkaIOReadsAndWritesCorrectlyInStreaming(KafkaIOIT.java:225)
1 test completed, 1 failed
Finished generating test XML results (0.026 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.029 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/reports/tests/integrationTest>
:sdks:java:io:kafka:integrationTest (Thread[Execution **** Thread 5,5,main]) completed. Took 32 mins 22.298 secs.
Resolve mutations for :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages (Thread[Execution **** Thread 7,5,main]) started.
Resolve mutations for :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages (Thread[Execution **** Thread 7,5,main]) completed. Took 0.0 secs.
:runners:google-cloud-dataflow-java:cleanUpDockerJavaImages (Thread[included builds,5,main]) started.
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Custom actions are attached to task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'.
Caching disabled for task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages' because:
Gradle would require more information to cache this task
Task ':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages' is not up-to-date because:
Task has not declared any outputs despite executing actions.
Starting process 'command 'docker''. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/runners/google-cloud-dataflow-java> Command: docker rmi --force us.gcr.io/apache-beam-testing/java-postcommit-it/java:20230412123722
Successfully started process 'command 'docker''
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20230412123722
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d80f92baad69968ed40df9e5568bfa94ff3de643f928dceae3d3229529337829
Starting process 'command 'gcloud''. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/runners/google-cloud-dataflow-java> Command: gcloud --quiet container images untag us.gcr.io/apache-beam-testing/java-postcommit-it/java:20230412123722
Successfully started process 'command 'gcloud''
WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly.
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20230412123722]
- referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d80f92baad69968ed40df9e5568bfa94ff3de643f928dceae3d3229529337829]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20230412123722] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d80f92baad69968ed40df9e5568bfa94ff3de643f928dceae3d3229529337829])].
Starting process 'command './scripts/cleanup_untagged_gcr_images.sh''. Working directory: <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/runners/google-cloud-dataflow-java> Command: ./scripts/cleanup_untagged_gcr_images.sh us.gcr.io/apache-beam-testing/java-postcommit-it/java
Successfully started process 'command './scripts/cleanup_untagged_gcr_images.sh''
Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d80f92baad69968ed40df9e5568bfa94ff3de643f928dceae3d3229529337829
Digests:
- us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d80f92baad69968ed40df9e5568bfa94ff3de643f928dceae3d3229529337829
Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:d80f92baad69968ed40df9e5568bfa94ff3de643f928dceae3d3229529337829].
:runners:google-cloud-dataflow-java:cleanUpDockerJavaImages (Thread[included builds,5,main]) completed. Took 4.69 secs.
Resolve mutations for :sdks:java:io:kafka:cleanUp (Thread[Execution **** Thread 5,5,main]) started.
Resolve mutations for :sdks:java:io:kafka:cleanUp (Thread[Execution **** Thread 5,5,main]) completed. Took 0.0 secs.
:sdks:java:io:kafka:cleanUp (Thread[Execution **** Thread 7,5,main]) started.
> Task :sdks:java:io:kafka:cleanUp
Skipping task ':sdks:java:io:kafka:cleanUp' as it has no actions.
:sdks:java:io:kafka:cleanUp (Thread[Execution **** Thread 7,5,main]) completed. Took 0.0 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:kafka:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/reports/tests/integrationTest/index.html>
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 33m 52s
164 actionable tasks: 103 executed, 57 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/mukp4nvtajuki
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PerformanceTests_Kafka_IO #3729
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/3729/display/redirect?page=changes>
Changes:
[noreply] [#25582] Use a Pathing Jar instead of long command line class paths in
------------------------------------------
[...truncated 993.27 KB...]
INFO: Adding Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues as step s14
Apr 12, 2023 1:16:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Values/Values/Map as step s15
Apr 12, 2023 1:16:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) as step s16
Apr 12, 2023 1:16:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) as step s17
Apr 12, 2023 1:16:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Create.Values/Read(CreateSource) as step s18
Apr 12, 2023 1:16:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Flatten.PCollections as step s19
Apr 12, 2023 1:16:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Window.Into()/Flatten.PCollections as step s20
Apr 12, 2023 1:16:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/WithKeys/AddKeys/Map as step s21
Apr 12, 2023 1:16:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/GroupByKey as step s22
Apr 12, 2023 1:16:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Values/Values/Map as step s23
Apr 12, 2023 1:16:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(Concat) as step s24
Apr 12, 2023 1:16:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GetPane/Map as step s25
Apr 12, 2023 1:16:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/RunChecks as step s26
Apr 12, 2023 1:16:52 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/VerifyAssertions/ParDo(DefaultConclude) as step s27
Apr 12, 2023 1:16:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.48.0-SNAPSHOT
Apr 12, 2023 1:16:53 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2023-04-11_18_16_52-11944772670229321344?project=apache-beam-testing
Apr 12, 2023 1:16:53 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2023-04-11_18_16_52-11944772670229321344
Apr 12, 2023 1:16:53 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2023-04-11_18_16_52-11944772670229321344
Apr 12, 2023 1:16:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2023-04-12T01:16:56.129Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: kafkaioit0testkafkaioreadsandwritescorrectlyinbatch-jenkin-cbg5. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions
Apr 12, 2023 1:17:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:07.724Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Apr 12, 2023 1:17:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:08.172Z: Worker configuration: e2-standard-2 in us-central1-b.
Apr 12, 2023 1:17:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:09.320Z: Expanding CoGroupByKey operations into optimizable parts.
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:09.434Z: Expanding GroupByKey operations into optimizable parts.
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:09.467Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:09.637Z: Fusing adjacent ParDo, Read, Write, and Flatten operations
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:09.668Z: Elided trivial flatten
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:09.695Z: Unzipping flatten s19 for input s17.org.apache.beam.sdk.values.PCollection.<init>:405#e63ad18dfc139ac0
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:09.727Z: Fusing unzipped copy of PAssert$0/GroupGlobally/WithKeys/AddKeys/Map, through flatten PAssert$0/GroupGlobally/Flatten.PCollections, into producer PAssert$0/GroupGlobally/ParDo(ToSingletonIterables)
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:09.762Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/Write into PAssert$0/GroupGlobally/GroupByKey/Reify
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:09.794Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/GroupByWindow into PAssert$0/GroupGlobally/GroupByKey/Read
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:09.828Z: Fusing consumer PAssert$0/GroupGlobally/Values/Values/Map into PAssert$0/GroupGlobally/GroupByKey/GroupByWindow
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:09.857Z: Fusing consumer PAssert$0/GroupGlobally/ParDo(Concat) into PAssert$0/GroupGlobally/Values/Values/Map
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:09.893Z: Fusing consumer PAssert$0/GetPane/Map into PAssert$0/GroupGlobally/ParDo(Concat)
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:09.918Z: Fusing consumer PAssert$0/RunChecks into PAssert$0/GetPane/Map
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:09.953Z: Fusing consumer PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:09.984Z: Unzipping flatten s19-u40 for input s21.org.apache.beam.sdk.values.PCollection.<init>:405#24aa17775bf242ac-c38
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.016Z: Fusing unzipped copy of PAssert$0/GroupGlobally/GroupByKey/Reify, through flatten PAssert$0/GroupGlobally/Flatten.PCollections/Unzipped-1, into producer PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.048Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Create/Read(CreateSource)
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.081Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.114Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.145Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.178Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.211Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Read
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.244Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.268Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.289Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Read into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.321Z: Fusing consumer Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/StripIds into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Read
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.341Z: Fusing consumer Measure read time into Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/StripIds
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.367Z: Fusing consumer Map records to strings/Map into Measure read time
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.394Z: Fusing consumer Calculate hashcode/WithKeys/AddKeys/Map into Map records to strings/Map
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.427Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate hashcode/WithKeys/AddKeys/Map
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.462Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.494Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.516Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.547Z: Fusing consumer Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.578Z: Fusing consumer Calculate hashcode/Values/Values/Map into Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.612Z: Fusing consumer PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous) into Calculate hashcode/Values/Values/Map
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.637Z: Fusing consumer PAssert$0/GroupGlobally/ParDo(ToSingletonIterables) into PAssert$0/GroupGlobally/Reify.Window/ParDo(Anonymous)
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.661Z: Fusing consumer PAssert$0/GroupGlobally/GroupByKey/Reify into PAssert$0/GroupGlobally/WithKeys/AddKeys/Map
Apr 12, 2023 1:17:11 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:10.694Z: Fusing consumer PAssert$0/GroupGlobally/WithKeys/AddKeys/Map into PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Apr 12, 2023 1:17:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:11.068Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Create
Apr 12, 2023 1:17:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:11.145Z: Starting 5 ****s in us-central1-b...
Apr 12, 2023 1:17:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:11.557Z: Finished operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Create
Apr 12, 2023 1:17:13 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:17:11.693Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Create/Read(CreateSource)+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write
Apr 12, 2023 1:18:03 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:18:02.862Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s).
Apr 12, 2023 1:18:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:18:31.363Z: Workers have started successfully.
Apr 12, 2023 1:18:52 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:18:52.055Z: All ****s have finished the startup processes and began to receive work requests.
Apr 12, 2023 1:18:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:18:56.259Z: Finished operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Create/Read(CreateSource)+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Split+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Pair with random key+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/Window.Into()/Window.Assign+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Reify+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Write
Apr 12, 2023 1:18:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:18:56.341Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Close
Apr 12, 2023 1:18:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:18:57.314Z: Finished operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Close
Apr 12, 2023 1:18:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:18:57.393Z: Executing operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Apr 12, 2023 1:18:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:18:57.556Z: Finished operation Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Apr 12, 2023 1:18:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2023-04-12T01:18:57.710Z: Executing operation Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/Read+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/GroupByKey/GroupByWindow+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Reshuffle/ExpandIterable+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Reshuffle/Values/Values/Map+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/Read+Read from bounded Kafka/KafkaIO.Read.ReadFromKafkaViaUnbounded/Read(KafkaUnboundedSource)/StripIds+Measure read time+Map records to strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Apr 12, 2023 1:46:53 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
WARNING: No terminal state was returned within allotted timeout. State value RUNNING
Apr 12, 2023 1:46:53 AM org.apache.kafka.common.config.AbstractConfig logAll
INFO: AdminClientConfig values:
bootstrap.servers = [34.173.204.97:32401, 34.72.86.171:32402, 35.193.176.184:32403]
client.dns.lookup = default
client.id =
connections.max.idle.ms = 300000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 120000
retries = 5
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
security.providers = null
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
Apr 12, 2023 1:46:53 AM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka version: 2.4.1
Apr 12, 2023 1:46:53 AM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka commitId: c57222ae8cd7866b
Apr 12, 2023 1:46:53 AM org.apache.kafka.common.utils.AppInfoParser$AppInfo <init>
INFO: Kafka startTimeMs: 1681264013628
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:io:kafka:integrationTest FAILED
org.apache.beam.sdk.io.kafka.KafkaIOIT > testKafkaIOReadsAndWritesCorrectlyInBatch FAILED
java.lang.AssertionError: expected:<DONE> but was:<null>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:120)
at org.junit.Assert.assertEquals(Assert.java:146)
at org.apache.beam.sdk.io.kafka.KafkaIOIT.testKafkaIOReadsAndWritesCorrectlyInBatch(KafkaIOIT.java:275)
1 test completed, 1 failed
Finished generating test XML results (0.032 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.049 secs) into: <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/reports/tests/integrationTest>
:sdks:java:io:kafka:integrationTest (Thread[Execution **** Thread 6,5,main]) completed. Took 37 mins 28.837 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:io:kafka:integrationTest'.
> There were failing tests. See the report at: file://<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/sdks/java/io/kafka/build/reports/tests/integrationTest/index.html>
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 38m 2s
143 actionable tasks: 1 executed, 142 up-to-date
Publishing build scan...
https://gradle.com/s/iw3oewjorq7ra
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org