[I] [Feature] Support limit the max snapshot count per checkpoint [paimon]

["Aitozi (via GitHub)" <gi...@apache.org>]

Aitozi opened a new issue, #5628:
URL: https://github.com/apache/paimon/issues/5628

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
   
   
   ### Motivation
   
   When running the streaming compaction job and catching up the delay, the compact source may consume much snapshot in one checkpoint, which lead to
   
   - The checkpoint time is too long, because the input compaction files too much
   - The delta committable is too big to hold in the CommitterOperator.
   
   We can limit the snapshot count per checkpoint to make the compaction job more smooth.
   
   ### Solution
   
   _No response_
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Feature] Support limit the max snapshot count per checkpoint [paimon]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi closed issue #5628: [Feature] Support limit the max snapshot count per checkpoint
URL: https://github.com/apache/paimon/issues/5628


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org