You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "xiangyuf (via GitHub)" <gi...@apache.org> on 2025/10/13 15:02:08 UTC
[PR] [FLINK-38510][table-planner] Remove the targetColumns from the digest generation of SinkReuser [flink]
xiangyuf opened a new pull request, #27107:
URL: https://github.com/apache/flink/pull/27107
## What is the purpose of the change
Remove the targetColumns from the digest generation of SinkReuser since it is deprecated now.
## Brief change log
Remove the targetColumns from the digest generation of SinkReuser.
## Verifying this change
This change added tests and can be verified as follows:
- *Added integration test in SinkReuseTestBase#testSinkReuseWithPartialColumnsNotSupportsTargetColumnWriting*
- *Added integration test in SinkReuseTestBase#testSinkReuseWithPartialColumnsAndSupportsTargetColumnWriting*
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? FLIP-506
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38510][table-planner] Remove targetColumns from the digest generation in SinkReuser [flink]
Posted by "xiangyuf (via GitHub)" <gi...@apache.org>.
xiangyuf commented on PR #27107:
URL: https://github.com/apache/flink/pull/27107#issuecomment-3421692856
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38510][table-planner] Remove the targetColumns from the digest generation of SinkReuser [flink]
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #27107:
URL: https://github.com/apache/flink/pull/27107#issuecomment-3397913583
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "445e4eb093a38a8bcc65d408ac5549975aeb7336",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "445e4eb093a38a8bcc65d408ac5549975aeb7336",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 445e4eb093a38a8bcc65d408ac5549975aeb7336 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38510][table-planner] Remove targetColumns from the digest generation of SinkReuser [flink]
Posted by "yunfengzhou-hub (via GitHub)" <gi...@apache.org>.
yunfengzhou-hub commented on PR #27107:
URL: https://github.com/apache/flink/pull/27107#issuecomment-3401496203
Thanks for the PR. LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38510][table-planner] Remove targetColumns from the digest generation of SinkReuser [flink]
Posted by "xiangyuf (via GitHub)" <gi...@apache.org>.
xiangyuf commented on code in PR #27107:
URL: https://github.com/apache/flink/pull/27107#discussion_r2434661265
##########
flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/batch/sql/BatchSinkReuseTest.xml:
##########
@@ -160,6 +160,35 @@ LogicalSink(table=[default_catalog.default_database.sink1], targetColumns=[[1]],
+- LogicalTableScan(table=[[default_catalog, default_database, source1]])
LogicalSink(table=[default_catalog.default_database.sink1], targetColumns=[[0]], fields=[x, EXPR$1])
++- LogicalProject(x=[$0], EXPR$1=[null:BIGINT])
+ +- LogicalTableScan(table=[[default_catalog, default_database, source3]])
+]]>
+ </Resource>
+ <Resource name="optimized exec plan">
+ <![CDATA[
+Sink(table=[default_catalog.default_database.sink1], targetColumns=[[0]], fields=[x, EXPR$1])
Review Comment:
Great idea. I think option1 looks good to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38510][table-planner] Remove targetColumns from the digest generation in SinkReuser [flink]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27107:
URL: https://github.com/apache/flink/pull/27107#issuecomment-3803350529
This PR is being marked as stale since it has not had any activity in the last 90 days.
If you would like to keep this PR alive, please leave a comment asking for a review.
If the PR has merge conflicts, update it with the latest from the base branch.
If you are having difficulty finding a reviewer, please reach out to the
community, contact details can be found here: https://flink.apache.org/what-is-flink/community/
If this PR is no longer valid or desired, please feel free to close it.
If no activity occurs in the next 30 days, it will be automatically closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38510][table-planner] Remove targetColumns from the digest generation in SinkReuser [flink]
Posted by "xiangyuf (via GitHub)" <gi...@apache.org>.
xiangyuf commented on PR #27107:
URL: https://github.com/apache/flink/pull/27107#issuecomment-3430181794
@xuyangzhong @yunfengzhou-hub CI passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38510][table-planner] Remove targetColumns from the digest generation of SinkReuser [flink]
Posted by "xuyangzhong (via GitHub)" <gi...@apache.org>.
xuyangzhong commented on code in PR #27107:
URL: https://github.com/apache/flink/pull/27107#discussion_r2430911342
##########
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/plan/common/SinkReuseTestBase.java:
##########
@@ -153,14 +150,25 @@ public void testSinkReuseFromSameSource() {
}
@Test
- public void testSinkReuseWithPartialColumns() {
+ public void testSinkReuseWithPartialColumnsNotSupportsTargetColumnWriting() {
StatementSet statementSet = util.tableEnv().createStatementSet();
+ /// sink1 has not implemented the {@link SupportsTargetColumnWriting} sink ability
Review Comment:
nit use `//` instead of using `///`
##########
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/plan/common/SinkReuseTestBase.java:
##########
@@ -153,14 +150,25 @@ public void testSinkReuseFromSameSource() {
}
@Test
- public void testSinkReuseWithPartialColumns() {
+ public void testSinkReuseWithPartialColumnsNotSupportsTargetColumnWriting() {
StatementSet statementSet = util.tableEnv().createStatementSet();
+ /// sink1 has not implemented the {@link SupportsTargetColumnWriting} sink ability
statementSet.addInsertSql("INSERT INTO sink1(`x`) (SELECT x FROM source1)");
statementSet.addInsertSql("INSERT INTO sink1(`y`) (SELECT y FROM source1)");
statementSet.addInsertSql("INSERT INTO sink1(`x`) (SELECT x FROM source3)");
util.verifyExecPlan(statementSet);
}
+ @Test
+ public void testSinkReuseWithPartialColumnsAndSupportsTargetColumnWriting() {
+ StatementSet statementSet = util.tableEnv().createStatementSet();
+ /// sink2 has implemented the {@link SupportsTargetColumnWriting} sink ability
Review Comment:
ditto
##########
flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/batch/sql/BatchSinkReuseTest.xml:
##########
@@ -160,6 +160,35 @@ LogicalSink(table=[default_catalog.default_database.sink1], targetColumns=[[1]],
+- LogicalTableScan(table=[[default_catalog, default_database, source1]])
LogicalSink(table=[default_catalog.default_database.sink1], targetColumns=[[0]], fields=[x, EXPR$1])
++- LogicalProject(x=[$0], EXPR$1=[null:BIGINT])
+ +- LogicalTableScan(table=[[default_catalog, default_database, source3]])
+]]>
+ </Resource>
+ <Resource name="optimized exec plan">
+ <![CDATA[
+Sink(table=[default_catalog.default_database.sink1], targetColumns=[[0]], fields=[x, EXPR$1])
Review Comment:
The plan is a bit strange with the description `targetColumns=[[0]]` in Sink.
I think we can either:
1. In the PhysicalSink, check whether SupportsTargetColumnWriting is present before printing targetColumns, and avoid printing targetColumns if it's not.
2. In the PhysicalSink, reference SupportsTargetColumnWriting when printing targetColumns, regardless of the targetColumns themselves.
3. Complete the printing of targetColumns based on https://issues.apache.org/jira/browse/FLINK-37708.
As of now, since targetColumns has not been removed from PhysicalSink, I tend to prefer option 1. What do you think, or do you have any other suggestions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38510][table-planner] Remove targetColumns from the digest generation of SinkReuser [flink]
Posted by "yunfengzhou-hub (via GitHub)" <gi...@apache.org>.
yunfengzhou-hub commented on PR #27107:
URL: https://github.com/apache/flink/pull/27107#issuecomment-3400227624
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38510][table-planner] Remove targetColumns from the digest generation in SinkReuser [flink]
Posted by "xuyangzhong (via GitHub)" <gi...@apache.org>.
xuyangzhong commented on code in PR #27107:
URL: https://github.com/apache/flink/pull/27107#discussion_r2469301188
##########
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/factories/TestUpdateDeleteTableFactory.java:
##########
@@ -447,6 +448,12 @@ public RowLevelUpdateMode getRowLevelUpdateMode() {
}
};
}
+
+ @Override
+ public boolean applyTargetColumns(int[][] targetColumns) {
Review Comment:
It seems a bit odd that the SupportsRowLevelUpdateSink test sink supports SupportsTargetColumnWriting but does nothing. I think it might not actually be necessary. 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38510][table-planner] Remove targetColumns from the digest generation in SinkReuser [flink]
Posted by "xiangyuf (via GitHub)" <gi...@apache.org>.
xiangyuf commented on PR #27107:
URL: https://github.com/apache/flink/pull/27107#issuecomment-3424846998
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38510][table-planner] Remove targetColumns from the digest generation of SinkReuser [flink]
Posted by "xiangyuf (via GitHub)" <gi...@apache.org>.
xiangyuf commented on PR #27107:
URL: https://github.com/apache/flink/pull/27107#issuecomment-3399693409
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-38510][table-planner] Remove targetColumns from the digest generation of SinkReuser [flink]
Posted by "xiangyuf (via GitHub)" <gi...@apache.org>.
xiangyuf commented on PR #27107:
URL: https://github.com/apache/flink/pull/27107#issuecomment-3397993710
@yunfengzhou-hub @xuyangzhong PTAL when u have time~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org