You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/02/24 02:44:55 UTC
[GitHub] [flink-table-store] tsreaper commented on pull request #24: [FLINK-26217] Introduce manifest.merge-min-count in commit
tsreaper commented on pull request #24:
URL: https://github.com/apache/flink-table-store/pull/24#issuecomment-1049439763
Hi @shenzhu ! Firstly thanks for your interest in contributing and also thanks @JingsongLi for the review.
I understand that we're introducing this configuration to optimize merging as well as to decrease the write overhead. However I'd like to suggest a better method for this.
Consider the merging process across snapshots. It's obvious that once a merged file exceeds the size limit it won't be merged again and will stay there forever. That is, the actual merging only happens at the tail of our manifest list. So why not just check the list from tail to head and once we hit a merged file we just exit? In this way we don't need this new configuration and we don't even need to iterate over the whole manifest list.
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org