You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "raminqaf (via GitHub)" <gi...@apache.org> on 2026/04/20 09:45:59 UTC

[PR] [FLINK-39479][python] Add fromChangelog() to Python Table API [flink]

raminqaf opened a new pull request, #27975:
URL: https://github.com/apache/flink/pull/27975

   ## What is the purpose of the change
   
   The Python Table API is missing the Table.fromChangelog() method introduced with the changelog PTFs (FLIP-564). This causes the TableAPICompletenessTests to fail after [FLINK-39419](https://issues.apache.org/jira/browse/FLINK-39479) added fromChangelog() to the Java Table interface.
   
   
   ## Brief change log
   
     - Add from_changelog() to pyflink/table/table.py
     - Remove fromChangelog from completeness test exclusions
   
   
   ## Verifying this change
   
   - Completeness check for python doesn't fail anymore
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): (no)
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes)
     - The serializers: (no)
     - The runtime per-record code paths (performance sensitive): (no)
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
     - The S3 file system connector: (no)
   
   ## Documentation
   
     - Does this pull request introduce a new feature? (yes)
     - If yes, how is the feature documented? (docs / comments)
     
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   <!--
   If generative AI tooling has been used in the process of authoring this PR, please
   change the checkbox below to `[X]` followed by the name of the tool, and uncomment the
   "Generated-by" line. See the ASF Generative Tooling Guidance for details:
   https://www.apache.org/legal/generative-tooling.html
   
   You are responsible for the quality and correctness of every change in this PR
   regardless of the tooling used. Low-effort AI-generated PRs will be closed. See
   AGENTS.md for the full guidance.
   -->
   
   - [ ] Yes (please specify the tool below)
   
   <!--
   Generated-by: [Tool Name and Version]
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-39479][python] Add fromChangelog() to Python Table API [flink]

Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #27975:
URL: https://github.com/apache/flink/pull/27975#issuecomment-4279573352

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "742968e9240928e08458b65194476015ecaf63ed",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "742968e9240928e08458b65194476015ecaf63ed",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 742968e9240928e08458b65194476015ecaf63ed UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-39479][python] Add fromChangelog() to Python Table API [flink]

Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #27975:
URL: https://github.com/apache/flink/pull/27975#discussion_r3110905047


##########
flink-python/pyflink/table/table.py:
##########
@@ -1220,6 +1220,40 @@ def to_changelog(self, *arguments: Expression) -> 'Table':
         """
         return Table(self._j_table.toChangelog(to_expression_jarray(arguments)), self._t_env)
 
+    def from_changelog(self, *arguments: Expression) -> 'Table':
+        """
+        Converts this append-only table with an explicit operation code column into a
+        dynamic table using the built-in ``FROM_CHANGELOG`` process table function.
+
+        Each input row is expected to have a string operation code column (default: ``op``)
+        that indicates the change operation (e.g., INSERT, UPDATE_AFTER, UPDATE_BEFORE,
+        DELETE). The output table is a dynamic table backed by a changelog stream.
+
+        Example:
+        ::
+
+            >>> from pyflink.table.expressions import descriptor, map_
+            >>> # Default: reads 'op' column with standard change operation names
+            >>> result = cdc_stream.from_changelog()

Review Comment:
   Done — both Python and Java docs now use the `result = ...` assignment style symmetrically across `to_changelog`/`toChangelog` and `from_changelog`/`fromChangelog`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-39479][python] Add fromChangelog() to Python Table API [flink]

Posted by "gustavodemorais (via GitHub)" <gi...@apache.org>.
gustavodemorais commented on code in PR #27975:
URL: https://github.com/apache/flink/pull/27975#discussion_r3110509283


##########
flink-python/pyflink/table/table.py:
##########
@@ -1220,6 +1220,40 @@ def to_changelog(self, *arguments: Expression) -> 'Table':
         """
         return Table(self._j_table.toChangelog(to_expression_jarray(arguments)), self._t_env)
 
+    def from_changelog(self, *arguments: Expression) -> 'Table':
+        """
+        Converts this append-only table with an explicit operation code column into a
+        dynamic table using the built-in ``FROM_CHANGELOG`` process table function.
+
+        Each input row is expected to have a string operation code column (default: ``op``)
+        that indicates the change operation (e.g., INSERT, UPDATE_AFTER, UPDATE_BEFORE,
+        DELETE). The output table is a dynamic table backed by a changelog stream.
+
+        Example:
+        ::
+
+            >>> from pyflink.table.expressions import descriptor, map_
+            >>> # Default: reads 'op' column with standard change operation names
+            >>> result = cdc_stream.from_changelog()

Review Comment:
   Same for the java docs



##########
flink-python/pyflink/table/table.py:
##########
@@ -1220,6 +1220,40 @@ def to_changelog(self, *arguments: Expression) -> 'Table':
         """
         return Table(self._j_table.toChangelog(to_expression_jarray(arguments)), self._t_env)
 
+    def from_changelog(self, *arguments: Expression) -> 'Table':
+        """
+        Converts this append-only table with an explicit operation code column into a
+        dynamic table using the built-in ``FROM_CHANGELOG`` process table function.
+
+        Each input row is expected to have a string operation code column (default: ``op``)
+        that indicates the change operation (e.g., INSERT, UPDATE_AFTER, UPDATE_BEFORE,
+        DELETE). The output table is a dynamic table backed by a changelog stream.
+
+        Example:
+        ::
+
+            >>> from pyflink.table.expressions import descriptor, map_
+            >>> # Default: reads 'op' column with standard change operation names
+            >>> result = cdc_stream.from_changelog()

Review Comment:
   We use result assignment (result = cdc_stream.from_changelog()) while the adjacent to_changelog method (line 1204) uses bare calls (table.to_changelog()). I think I like yours better. Can you update it so both are symmetric?  



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-39479][python] Add fromChangelog() to Python Table API [flink]

Posted by "twalthr (via GitHub)" <gi...@apache.org>.
twalthr merged PR #27975:
URL: https://github.com/apache/flink/pull/27975


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-39479][python] Add fromChangelog() to Python Table API [flink]

Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #27975:
URL: https://github.com/apache/flink/pull/27975#discussion_r3110900813


##########
flink-python/pyflink/table/table.py:
##########
@@ -1220,6 +1220,40 @@ def to_changelog(self, *arguments: Expression) -> 'Table':
         """
         return Table(self._j_table.toChangelog(to_expression_jarray(arguments)), self._t_env)
 
+    def from_changelog(self, *arguments: Expression) -> 'Table':
+        """
+        Converts this append-only table with an explicit operation code column into a
+        dynamic table using the built-in ``FROM_CHANGELOG`` process table function.
+
+        Each input row is expected to have a string operation code column (default: ``op``)
+        that indicates the change operation (e.g., INSERT, UPDATE_AFTER, UPDATE_BEFORE,
+        DELETE). The output table is a dynamic table backed by a changelog stream.
+
+        Example:
+        ::
+
+            >>> from pyflink.table.expressions import descriptor, map_
+            >>> # Default: reads 'op' column with standard change operation names
+            >>> result = cdc_stream.from_changelog()

Review Comment:
   Done — both Python and Java docs now use the  assignment style symmetrically across / and /.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-39479][python] Add fromChangelog() to Python Table API [flink]

Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #27975:
URL: https://github.com/apache/flink/pull/27975#discussion_r3116040072


##########
flink-python/pyflink/table/table.py:
##########
@@ -1220,6 +1220,40 @@ def to_changelog(self, *arguments: Expression) -> 'Table':
         """
         return Table(self._j_table.toChangelog(to_expression_jarray(arguments)), self._t_env)
 
+    def from_changelog(self, *arguments: Expression) -> 'Table':
+        """
+        Converts this append-only table with an explicit operation code column into a
+        dynamic table using the built-in ``FROM_CHANGELOG`` process table function.
+
+        Each input row is expected to have a string operation code column (default: ``op``)
+        that indicates the change operation (e.g., INSERT, UPDATE_AFTER, UPDATE_BEFORE,
+        DELETE). The output table is a dynamic table backed by a changelog stream.

Review Comment:
   Sure! updated both docs in a new commit



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-39479][python] Add fromChangelog() to Python Table API [flink]

Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #27975:
URL: https://github.com/apache/flink/pull/27975#discussion_r3116065511


##########
flink-python/pyflink/table/table.py:
##########
@@ -1220,6 +1220,40 @@ def to_changelog(self, *arguments: Expression) -> 'Table':
         """
         return Table(self._j_table.toChangelog(to_expression_jarray(arguments)), self._t_env)
 
+    def from_changelog(self, *arguments: Expression) -> 'Table':

Review Comment:
   @Au-Miner Thanks for pointing this out. Added a new ticket so we can track this: https://issues.apache.org/jira/browse/FLINK-39505



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-39479][python] Add fromChangelog() to Python Table API [flink]

Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #27975:
URL: https://github.com/apache/flink/pull/27975#discussion_r3110900813


##########
flink-python/pyflink/table/table.py:
##########
@@ -1220,6 +1220,40 @@ def to_changelog(self, *arguments: Expression) -> 'Table':
         """
         return Table(self._j_table.toChangelog(to_expression_jarray(arguments)), self._t_env)
 
+    def from_changelog(self, *arguments: Expression) -> 'Table':
+        """
+        Converts this append-only table with an explicit operation code column into a
+        dynamic table using the built-in ``FROM_CHANGELOG`` process table function.
+
+        Each input row is expected to have a string operation code column (default: ``op``)
+        that indicates the change operation (e.g., INSERT, UPDATE_AFTER, UPDATE_BEFORE,
+        DELETE). The output table is a dynamic table backed by a changelog stream.
+
+        Example:
+        ::
+
+            >>> from pyflink.table.expressions import descriptor, map_
+            >>> # Default: reads 'op' column with standard change operation names
+            >>> result = cdc_stream.from_changelog()

Review Comment:
   Done — both Python and Java docs now use the  assignment style symmetrically across / and /.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-39479][python] Add fromChangelog() to Python Table API [flink]

Posted by "twalthr (via GitHub)" <gi...@apache.org>.
twalthr commented on code in PR #27975:
URL: https://github.com/apache/flink/pull/27975#discussion_r3115894222


##########
flink-python/pyflink/table/table.py:
##########
@@ -1220,6 +1220,40 @@ def to_changelog(self, *arguments: Expression) -> 'Table':
         """
         return Table(self._j_table.toChangelog(to_expression_jarray(arguments)), self._t_env)
 
+    def from_changelog(self, *arguments: Expression) -> 'Table':
+        """
+        Converts this append-only table with an explicit operation code column into a
+        dynamic table using the built-in ``FROM_CHANGELOG`` process table function.
+
+        Each input row is expected to have a string operation code column (default: ``op``)
+        that indicates the change operation (e.g., INSERT, UPDATE_AFTER, UPDATE_BEFORE,
+        DELETE). The output table is a dynamic table backed by a changelog stream.

Review Comment:
   ```
   Converts this append-only table with an explicit operation code column into a (potentially updating) dynamic table. Each input row is expected to have a string column that indicates the change operation. The operation column is interpreted by the engine and removed from the output.
   ```
   Can we stay closer to the website docs we agreed upon? Maybe also update JavaDoc accordingly.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-39479][python] Add fromChangelog() to Python Table API [flink]

Posted by "twalthr (via GitHub)" <gi...@apache.org>.
twalthr commented on code in PR #27975:
URL: https://github.com/apache/flink/pull/27975#discussion_r3115871873


##########
flink-python/pyflink/table/table.py:
##########
@@ -1220,6 +1220,40 @@ def to_changelog(self, *arguments: Expression) -> 'Table':
         """
         return Table(self._j_table.toChangelog(to_expression_jarray(arguments)), self._t_env)
 
+    def from_changelog(self, *arguments: Expression) -> 'Table':

Review Comment:
   Make sense. Overall Python is only forwarding to Java, so we did not always adds tests for the conversion. If the file syntactically compiles there is a high chance that also a test should pass.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-39479][python] Add fromChangelog() to Python Table API [flink]

Posted by "Au-Miner (via GitHub)" <gi...@apache.org>.
Au-Miner commented on code in PR #27975:
URL: https://github.com/apache/flink/pull/27975#discussion_r3111655134


##########
flink-python/pyflink/table/table.py:
##########
@@ -1220,6 +1220,40 @@ def to_changelog(self, *arguments: Expression) -> 'Table':
         """
         return Table(self._j_table.toChangelog(to_expression_jarray(arguments)), self._t_env)
 
+    def from_changelog(self, *arguments: Expression) -> 'Table':

Review Comment:
   Currently, both the from_changelog and to_changelog lack corresponding Python side tests, and have even been enlarged to include insert_into and print_explain in the entire table.py. I think we can create a Jira to fill in this part of the testing.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org