You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2022/07/07 07:59:45 UTC
[GitHub] [rocketmq] lizhanhui opened a new issue, #4568: Improve performance to ConsumeQueue when reclaiming disk space
lizhanhui opened a new issue, #4568:
URL: https://github.com/apache/rocketmq/issues/4568
When commit log files are deleted, **ALL** consume queues scan its last file linearly, from beginning to its written position. RocketMQ is designed to serve a large number of topics, thus an even bigger number of consume-queues... This style of scan becomes very inefficient when these numbers grow larger and system memory is tight.
We recently observed this correction procedure takes a significant amount of IO bandwidth, impacting QoS noticeably in production.
A few improvements can be made to mitigate/resolve this issue observed.
1. For topics/consume queues that are deprecated, this scan should be zero overhead;
2. Scan should start from the previous min-offset wherever possible, instead of the beginning of the first file;
3. The scan should employ binary search as entries in consume queue are naturally sorted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] duhenglucky closed issue #4568: Improve performance to ConsumeQueue when reclaiming disk space
Posted by GitBox <gi...@apache.org>.
duhenglucky closed issue #4568: Improve performance to ConsumeQueue when reclaiming disk space
URL: https://github.com/apache/rocketmq/issues/4568
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] lizhanhui commented on issue #4568: Improve performance to ConsumeQueue when reclaiming disk space
Posted by GitBox <gi...@apache.org>.
lizhanhui commented on issue #4568:
URL: https://github.com/apache/rocketmq/issues/4568#issuecomment-1178456711
@duhenglucky Regarding the release plan. It's known that RocketMQ has released its LTS version a few days ago. However, LTS normally has follow-up minor updates to further consolidate its support. For example, Ubuntu has very good release cycle management. See https://wiki.ubuntu.com/Releases
After releasing Ubuntu 20.04 LTS, it further released Ubuntu 20.04.1 LTS, Ubuntu 20.04.2 LTS Ubuntu 20.04.3 LTS Ubuntu 20.04.4 LTS to demonstrate continuous support.
With that said, shall we have a release plan, like 4.9.4.1 LTS, to accept fixes similar to this pull request?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org