You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pegasus.apache.org by "empiredan (via GitHub)" <gi...@apache.org> on 2023/10/16 07:10:16 UTC
[PR] feat(slog): flush and remove all shared logs for garbage collection [incubator-pegasus]
empiredan opened a new pull request, #1638:
URL: https://github.com/apache/incubator-pegasus/pull/1638
https://github.com/apache/incubator-pegasus/issues/1593
In https://github.com/XiaoMi/rdsn/pull/1019 we've written private logs as
WAL instead of shared logs, which also means shared log files would never
be appended with new mutations.
However, obsolete shared logs that had been applied to rocksdb were not
removed since then. There is at least 1 shared log file which is never removed.
We should change this policy of garbage collection, to delete all obsolete
shared log files.
To facilitate the garbage collection of shared log files, we also trigger checkpoints
which would flush rocksdb data to disk for each replica that has prevented shared
log files from being removed for garbage collection. It is necessary to limit the
number of submitted replicas that would be flushed at a time to avoid too much
consumption of I/O resources which might affect the processing of read/write
requests from clients.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
Re: [PR] feat(slog): flush and remove all shared logs for garbage collection (cherry-pick #1594) [incubator-pegasus]
Posted by "acelyc111 (via GitHub)" <gi...@apache.org>.
acelyc111 merged PR #1638:
URL: https://github.com/apache/incubator-pegasus/pull/1638
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org