You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "huyuanfeng2018 (via GitHub)" <gi...@apache.org> on 2025/09/10 07:15:22 UTC
[PR] [core] Support push down branchesTable by branchName [paimon]
huyuanfeng2018 opened a new pull request, #6231:
URL: https://github.com/apache/paimon/pull/6231
<!-- Please specify the module before the PR name: [core] ... or [flink] ... -->
### Purpose
<!-- Linking this pull request to the issue -->
Linked issue: close #xxx
In Paimon's system table $branches, when a user needs to query specific branch information, the previous implementation did not support pushing down the filtering criteria of branch_name.
This means that even if we only want to query one or a few specific branches, Paimon will read all branch information and then filter it at the computation engine layer
This change aims to optimize the query performance of the $branches table by implementing the pushdown of filtering conditions for the branch_name field, allowing queries to be filtered directly at the storage layer based on the branch name, thereby significantly improving query efficiency.
<!-- What is the purpose of the change -->
### Tests
<!-- List UT and IT cases to verify this change -->
org.apache.paimon.flink.BranchSqlITCase##testBranchesTableFilter with adding more case for the method.
### API and Format
<!-- Does this change affect API or storage format -->
### Documentation
<!-- Does this change introduce a new feature -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Support push down branchesTable by branchName [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #6231:
URL: https://github.com/apache/paimon/pull/6231
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Support push down branchesTable by branchName [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #6231:
URL: https://github.com/apache/paimon/pull/6231#issuecomment-3274219160
> @JingsongLi Can take some time to help me review this PR? Thank you so much~
Thanks @huyuanfeng2018 for the contribution. Do your scenes have many branches? I rarely see situations where there are many branches, so I haven't optimized it here either.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Support push down branchesTable by branchName [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #6231:
URL: https://github.com/apache/paimon/pull/6231#issuecomment-3274359376
> > > @JingsongLi Can take some time to help me review this PR? Thank you so much~
> >
> >
> > Thanks @huyuanfeng2018 for the contribution. Do your scenes have many branches? I rarely see situations where there are many branches, so I haven't optimized it here either.
>
> When using cdc synchronization, we generate paimon branch every hour accurately split hourly snapshots (tags can't do that), so there will be many branch
Got it, I will take a look~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Support push down branchesTable by branchName [paimon]
Posted by "huyuanfeng2018 (via GitHub)" <gi...@apache.org>.
huyuanfeng2018 commented on PR #6231:
URL: https://github.com/apache/paimon/pull/6231#issuecomment-3274202683
@JingsongLi Can take some time to help me review this PR? Thank you so much~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Support push down branchesTable by branchName [paimon]
Posted by "huyuanfeng2018 (via GitHub)" <gi...@apache.org>.
huyuanfeng2018 commented on PR #6231:
URL: https://github.com/apache/paimon/pull/6231#issuecomment-3274320911
> > @JingsongLi Can take some time to help me review this PR? Thank you so much~
>
> Thanks @huyuanfeng2018 for the contribution. Do your scenes have many branches? I rarely see situations where there are many branches, so I haven't optimized it here either.
When using cdc synchronization, we generate paimon branch every hour accurately split hourly snapshots (tags can't do that), so there will be many branch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org