You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "KenanAdel (via GitHub)" <gi...@apache.org> on 2026/06/15 23:37:07 UTC

[PR] [FLINK-38918][PyFlink] Support per cluster offset for DynamicKafkaSource in pyflink [flink]

KenanAdel opened a new pull request, #28457:
URL: https://github.com/apache/flink/pull/28457

   ## What is the purpose of the change
   
   This PR adds support for per-cluster starting and stopping offset initializers in PyFlink's `DynamicKafkaSource`, aligning the Python API with the underlying Java implementation introduced in FLINK-38876.
   
   ## Brief change log
   
   - Updated `ClusterMetadata` and `SingleClusterTopicMetadataService` constructors in `dynamic_kafka.py` to accept optional `starting_offsets_initializer` and `stopping_offsets_initializer`.
   - Forwarded the PyFlink offset initializers to the corresponding Java objects via the Py4J gateway using `_j_initializer`.
   - Ensured proper propagation of offset configuration from Python layer to the underlying Java metadata service.
   - Added comprehensive unit tests in `test_dynamic_kafka.py` to verify offset forwarding and ensure backward compatibility when offsets are not provided.
   
   ## Verifying this change
   
   This change is covered by the following newly added unit tests in `test_dynamic_kafka.py`:
   - `test_single_cluster_metadata_service_with_offsets`
   - `test_single_cluster_metadata_service_default_offsets`
   - `test_cluster_metadata_with_offsets`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-38918][PyFlink] Support per cluster offset for DynamicKafkaSource in pyflink [flink]

Posted by "KenanAdel (via GitHub)" <gi...@apache.org>.
KenanAdel closed pull request #28457: [FLINK-38918][PyFlink] Support per cluster offset for DynamicKafkaSource in pyflink
URL: https://github.com/apache/flink/pull/28457


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-38918][PyFlink] Support per cluster offset for DynamicKafkaSource in pyflink [flink]

Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #28457:
URL: https://github.com/apache/flink/pull/28457#issuecomment-4713427800

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ba0165742c98cf605f31a60a0c7be4ca3cfc4469",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "ba0165742c98cf605f31a60a0c7be4ca3cfc4469",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ba0165742c98cf605f31a60a0c7be4ca3cfc4469 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org