[jira] [Created] (FLINK-40064) Migrate PyFlink DataStream e2e test from Kafka to FileSource/FileSink and re-enable it

["Martijn Visser (Jira)" <ji...@apache.org>]

Martijn Visser created FLINK-40064:
--------------------------------------

             Summary: Migrate PyFlink DataStream e2e test from Kafka to FileSource/FileSink and re-enable it
                 Key: FLINK-40064
                 URL: https://issues.apache.org/jira/browse/FLINK-40064
             Project: Flink
          Issue Type: Bug
          Components: API / Python, Test Infrastructure
            Reporter: Martijn Visser
            Assignee: Martijn Visser


FLINK-40048 removed flink-sql-connector-kafka from flink-sql-client-test, causing test_pyflink.sh to fail on every run. The only consumer of that jar was the "Test PyFlink DataStream job" case, disabled since FLINK-36185 because data_stream_job.py still used the legacy FlinkKafkaConsumer/FlinkKafkaProducer.

We should rewrite data_stream_job.py to the in-repo FileSource (text_line_format) and unified FileSink, re-enable the case with deterministic file-based verification (auto-watermark interval 0, bounded source, job runs to FINISHED), and remove the Kafka scaffolding from test_pyflink.sh. kafka_sql_common.sh remains (used by test_confluent_schema_registry.sh). Restores the nightly leg and PyFlink DataStream KeyedProcessFunction/event-time-timer coverage dormant since Sept 2024.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)