You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "zhoulii (via GitHub)" <gi...@apache.org> on 2025/11/25 12:48:46 UTC
[I] [Imporvement] Avoid scanning all snapshots to find the latest snapshot of user for batch operations. [paimon]
zhoulii opened a new issue, #6671:
URL: https://github.com/apache/paimon/issues/6671
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Motivation
When a Paimon table retains a large number of snapshots (e.g., due to long retention periods), batch commit may slow down by the filterCommitted method of FileStoreCommitImpl, which calls latestSnapshotOfUser to find the latest snapshot for a specific commit user. This method may need to scan through all snapshots sequentially when there are many snapshots, causing unacceptable delays for batch commits.
### Solution
skip the latestSnapshotOfUser call for batch commit.
### Anything else?
_No response_
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Imporvement] Avoid scanning all snapshots to find the latest snapshot of user for batch operations. [paimon]
Posted by "zhoulii (via GitHub)" <gi...@apache.org>.
zhoulii closed issue #6671: [Imporvement] Avoid scanning all snapshots to find the latest snapshot of user for batch operations.
URL: https://github.com/apache/paimon/issues/6671
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org