You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "lizhimins (via GitHub)" <gi...@apache.org> on 2026/04/21 12:26:19 UTC
[PR] [ISSUE #10270] Make Pop RocksDB BlockCache size configurable via MessageStoreConfig [rocketmq]
lizhimins opened a new pull request, #10271:
URL: https://github.com/apache/rocketmq/pull/10271
Add popRocksdbBlockCacheSize to MessageStoreConfig with a default of 256MB, replacing the hardcoded value in RocksDBOptionsFactory.createPopCFOptions(). Default ColumnFamily uses a minimal 16MB cache since it stores no meaningful data. This allows operators to tune Pop RocksDB memory usage based on container memory budgets.
<!-- Please make sure the target branch is right. In most case, the target branch should be `develop`. -->
### Which Issue(s) This PR Fixes
<!-- Please ensure that the related issue has already been created, and [link this pull request to that issue using keywords](<https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword>) to ensure automatic closure. -->
- Fixes #10270
### Brief Description
<!-- Write a brief description for your pull request to help the maintainer understand the reasons behind your changes. -->
### How Did You Test This Change?
<!-- In order to ensure the code quality of Apache RocketMQ, we expect every pull request to have undergone thorough testing. -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ISSUE #10270] Make Pop RocksDB BlockCache size configurable via MessageStoreConfig [rocketmq]
Posted by "qianye1001 (via GitHub)" <gi...@apache.org>.
qianye1001 commented on PR #10271:
URL: https://github.com/apache/rocketmq/pull/10271#issuecomment-4293206879
Thanks for making the Pop RocksDB BlockCache size configurable — this is a good step toward better memory tuning in container environments.
One thing worth considering: `blockCacheSize` is only part of the memory footprint. In `createPopCFOptions()`, `writeBufferSize` (128 MB) and `maxWriteBufferNumber` (4) are still hardcoded, which means the memtable alone can consume up to 512 MB. For memory-constrained containers, this is just as significant as the block cache.
It would be great to also expose `writeBufferSize` and `maxWriteBufferNumber` (and possibly `minWriteBufferNumberToMerge`) as configurable fields in `MessageStoreConfig`, so operators can holistically tune the Pop RocksDB memory budget. Otherwise, someone who carefully tunes the block cache down may still be surprised by the fixed memtable overhead.
Happy to discuss further if needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ISSUE #10270] Make Pop RocksDB BlockCache size configurable via MessageStoreConfig [rocketmq]
Posted by "lizhimins (via GitHub)" <gi...@apache.org>.
lizhimins merged PR #10271:
URL: https://github.com/apache/rocketmq/pull/10271
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ISSUE #10270] Make Pop RocksDB BlockCache size configurable via MessageStoreConfig [rocketmq]
Posted by "qianye1001 (via GitHub)" <gi...@apache.org>.
qianye1001 commented on PR #10271:
URL: https://github.com/apache/rocketmq/pull/10271#issuecomment-4294131990
LGTM, thanks for the quick update!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ISSUE #10270] Make Pop RocksDB BlockCache size configurable via MessageStoreConfig [rocketmq]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #10271:
URL: https://github.com/apache/rocketmq/pull/10271#issuecomment-4288701683
## [Codecov](https://app.codecov.io/gh/apache/rocketmq/pull/10271?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `60.00000%` with `4 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.86%. Comparing base ([`f04eafc`](https://app.codecov.io/gh/apache/rocketmq/commit/f04eafcd7b0d9739e976e308f4f87a5bde033c78?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f5943a4`](https://app.codecov.io/gh/apache/rocketmq/commit/f5943a43cbdc662e2651ece2eeacb9da9e7d66c2?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/rocketmq/pull/10271?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ache/rocketmq/store/config/MessageStoreConfig.java](https://app.codecov.io/gh/apache/rocketmq/pull/10271?src=pr&el=tree&filepath=store%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fstore%2Fconfig%2FMessageStoreConfig.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL2NvbmZpZy9NZXNzYWdlU3RvcmVDb25maWcuamF2YQ==) | 25.00% | [3 Missing :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/10271?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [.../rocketmq/store/rocksdb/RocksDBOptionsFactory.java](https://app.codecov.io/gh/apache/rocketmq/pull/10271?src=pr&el=tree&filepath=store%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fstore%2Frocksdb%2FRocksDBOptionsFactory.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL3JvY2tzZGIvUm9ja3NEQk9wdGlvbnNGYWN0b3J5LmphdmE=) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/10271?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## develop #10271 +/- ##
=============================================
- Coverage 48.89% 48.86% -0.03%
+ Complexity 13443 13439 -4
=============================================
Files 1375 1375
Lines 100436 100442 +6
Branches 12971 12971
=============================================
- Hits 49107 49083 -24
- Misses 45320 45352 +32
+ Partials 6009 6007 -2
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/rocketmq/pull/10271?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org