Beam High Priority Issue Report (65)

[be...@gmail.com]

This is your daily summary of Beam's current high priority issues that may need attention.

    See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities.

Unassigned P1 Issues:

https://github.com/apache/beam/issues/30530 The LoadTests Java GBK Smoke job is flaky
https://github.com/apache/beam/issues/30529 The PostCommit Java Sickbay job is flaky
https://github.com/apache/beam/issues/30528 The Dask Runner Tests job is flaky
https://github.com/apache/beam/issues/30527 The PostCommit Java IO Performance Tests job is flaky
https://github.com/apache/beam/issues/30526 The PerformanceTests xlang KafkaIO Python job is flaky
https://github.com/apache/beam/issues/30525 The PostCommit Python ValidatesContainer Dataflow With RC job is flaky
https://github.com/apache/beam/issues/30524 The PerformanceTests SingleStoreIO job is flaky
https://github.com/apache/beam/issues/30523 The PostCommit Java SingleStoreIO IT job is flaky
https://github.com/apache/beam/issues/30522 The Build and Version Runner Docker Image job is flaky
https://github.com/apache/beam/issues/30521 The LoadTests Go Combine Flink Batch job is flaky
https://github.com/apache/beam/issues/30520 The LoadTests Python Combine Flink Streaming job is flaky
https://github.com/apache/beam/issues/30519 The PostCommit XVR GoUsingJava Dataflow job is flaky
https://github.com/apache/beam/issues/30518 The Update Python Depedencies job is flaky
https://github.com/apache/beam/issues/30517 The PostCommit XVR Direct job is flaky
https://github.com/apache/beam/issues/30516 The PostCommit Java ValidatesRunner SparkStructuredStreaming job is flaky
https://github.com/apache/beam/issues/30515 The LoadTests Go SideInput Flink Batch job is flaky
https://github.com/apache/beam/issues/30514 The Clean Up Prebuilt SDK Images job is flaky
https://github.com/apache/beam/issues/30513 The PostCommit Python job is flaky
https://github.com/apache/beam/issues/30512 The PostCommit Java PVR Spark Batch job is flaky
https://github.com/apache/beam/issues/30511 The LoadTests Python Smoke job is flaky
https://github.com/apache/beam/issues/30510 The PostCommit Java InfluxDbIO Integration Test job is flaky
https://github.com/apache/beam/issues/30508 The PostCommit Java ValidatesRunner Flink job is flaky
https://github.com/apache/beam/issues/30507 The LoadTests Go GBK Flink Batch job is flaky
https://github.com/apache/beam/issues/30506 The TypeScript Tests job is flaky
https://github.com/apache/beam/issues/30505 The PostRelease Nightly Snapshot job is flaky
https://github.com/apache/beam/issues/30504 The LoadTests Python Combine Dataflow Streaming job is flaky
https://github.com/apache/beam/issues/30503 The PostCommit Java ValidatesRunner Flink Java11 job is flaky
https://github.com/apache/beam/issues/30502 The LoadTests Go CoGBK Flink Batch job is flaky
https://github.com/apache/beam/issues/30498 [Bug]: Beam Sql is ignoring aliases fields in some situations which causes to huge data loss
https://github.com/apache/beam/issues/30459 [Failing Test]: Python PostCommit failing hdfsIntegrationTest in generate_external_transform_wrappers
https://github.com/apache/beam/issues/30447 [Failing Test]: beam_PostRelease_NightlySnapshot red due to various flakes in Dataflow example tests
https://github.com/apache/beam/issues/29971 [Bug]: FixedWindows not working for large Kafka topic
https://github.com/apache/beam/issues/29926 [Bug]: FileIO: lack of timeouts may cause the pipeline to get stuck indefinitely
https://github.com/apache/beam/issues/29902 [Bug]: Messages are not ACK on Pubsub starting Beam 2.52.0 on Flink Runner in detached mode
https://github.com/apache/beam/issues/29099 [Bug]: FnAPI Java SDK Harness doesn't update user counters in OnTimer callback functions
https://github.com/apache/beam/issues/28760 [Bug]: EFO Kinesis IO reader provided by apache beam does not pick the event time for watermarking
https://github.com/apache/beam/issues/28383 [Failing Test]: org.apache.beam.runners.dataflow.worker.StreamingDataflowWorkerTest.testMaxThreadMetric
https://github.com/apache/beam/issues/28326 Bug: apache_beam.io.gcp.pubsublite.ReadFromPubSubLite not working
https://github.com/apache/beam/issues/27892 [Bug]: ignoreUnknownValues not working when using CreateDisposition.CREATE_IF_NEEDED 
https://github.com/apache/beam/issues/27616 [Bug]: Unable to use applyRowMutations() in bigquery IO apache beam java
https://github.com/apache/beam/issues/27486 [Bug]: Read from datastore with inequality filters
https://github.com/apache/beam/issues/27314 [Failing Test]: bigquery.StorageApiSinkCreateIfNeededIT.testCreateManyTables[1]
https://github.com/apache/beam/issues/27238 [Bug]: Window trigger has lag when using Kafka and GroupByKey on Dataflow Runner
https://github.com/apache/beam/issues/26911 [Bug]: UNNEST ARRAY with a nested ROW (described below)
https://github.com/apache/beam/issues/26343 [Bug]: apache_beam.io.gcp.bigquery_read_it_test.ReadAllBQTests.test_read_queries is flaky
https://github.com/apache/beam/issues/26329 [Bug]: BigQuerySourceBase does not propagate a Coder to AvroSource
https://github.com/apache/beam/issues/26041 [Bug]: Unable to create exactly-once Flink pipeline with stream source and file sink
https://github.com/apache/beam/issues/24776 [Bug]: Race condition in Python SDK Harness ProcessBundleProgress
https://github.com/apache/beam/issues/24313 [Flaky]: apache_beam/runners/portability/portable_runner_test.py::PortableRunnerTestWithSubprocesses::test_pardo_state_with_custom_key_coder
https://github.com/apache/beam/issues/23709 [Flake]: Spark batch flakes in ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElement and ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundle
https://github.com/apache/beam/issues/23525 [Bug]: Default PubsubMessage coder will drop message id and orderingKey
https://github.com/apache/beam/issues/22913 [Bug]: beam_PostCommit_Java_ValidatesRunner_Flink is flakes in org.apache.beam.sdk.transforms.GroupByKeyTest$BasicTests.testAfterProcessingTimeContinuationTriggerUsingState
https://github.com/apache/beam/issues/22605 [Bug]: Beam Python failure for dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest.test_metrics_it
https://github.com/apache/beam/issues/21706 Flaky timeout in github Python unit test action StatefulDoFnOnDirectRunnerTest.test_dynamic_timer_clear_then_set_timer
https://github.com/apache/beam/issues/21643 FnRunnerTest with non-trivial (order 1000 elements) numpy input flakes in non-cython environment
https://github.com/apache/beam/issues/21476 WriteToBigQuery Dynamic table destinations returns wrong tableId
https://github.com/apache/beam/issues/21260 Python DirectRunner does not emit data at GC time
https://github.com/apache/beam/issues/21121 apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT.test_streaming_wordcount_it flakey
https://github.com/apache/beam/issues/21104 Flaky: apache_beam.runners.portability.fn_api_runner.fn_runner_test.FnApiRunnerTestWithGrpcAndMultiWorkers
https://github.com/apache/beam/issues/20976 apache_beam.runners.portability.flink_runner_test.FlinkRunnerTestOptimized.test_flink_metrics is flaky
https://github.com/apache/beam/issues/20108 Python direct runner doesn't emit empty pane when it should
https://github.com/apache/beam/issues/19814 Flink streaming flakes in ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundleStateful and ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElementStateful


P1 Issues with no update in the last week:

https://github.com/apache/beam/issues/29515 [Bug]: WriteToFiles in python leave few records in temp directory when writing to large number (100+) of files
https://github.com/apache/beam/issues/27022 [Bug]: Possible data loss in BigtableIO r/w if timestamp not set (default to epoch)
https://github.com/apache/beam/issues/25975 [Bug]: KinesisIO processing-time watermarking can cause data loss