You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "RongtongJin (via GitHub)" <gi...@apache.org> on 2025/09/14 09:32:37 UTC
[PR] [ISSUE #9699] Optimize shutdown process and resource management [rocketmq]
RongtongJin opened a new pull request, #9700:
URL: https://github.com/apache/rocketmq/pull/9700
<!-- Please make sure the target branch is right. In most case, the target branch should be `develop`. -->
### Which Issue(s) This PR Fixes
<!-- Please ensure that the related issue has already been created, and [link this pull request to that issue using keywords](<https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword>) to ensure automatic closure. -->
Fixes #9699
This PR optimizes the shutdown process and resource management across multiple RocketMQ components to ensure graceful shutdown and prevent resource leaks.
### Brief Description
<!-- Write a brief description for your pull request to help the maintainer understand the reasons behind your changes. -->
**Key Improvements:**
1. **Enhanced BrokerController Shutdown Sequence**
- Improved shutdown order for all broker services
- Added proper termination for transactional message services
- Enhanced cleanup of notification processors and offset managers
2. **Storage Layer Optimizations**
- **DefaultMessageStore**: Enhanced shutdown flow with proper state management
- **CommitLog**: Improved resource cleanup during shutdown
- **MappedFileQueue**: Added proper shutdown handling
- **StoreCheckpoint**: Enhanced checkpoint shutdown process
- **DefaultHAService**: Improved HA service shutdown
3. **Timer Component Enhancements**
- **TimerMessageStore**: Comprehensive shutdown improvements with proper resource cleanup
- **TimerCheckpoint**: Enhanced checkpoint shutdown process
- **TimerLog**: Improved log shutdown handling
- **TimerWheel**: Better wheel shutdown management
4. **Processor Improvements**
- **AckMessageProcessor**: Added proper shutdown handling
- **NotificationProcessor**: Enhanced shutdown process
- **PopMessageProcessor**: Improved shutdown sequence
- **PopBufferMergeService**: Better resource cleanup
5. **Common Components**
- **AbstractRocksDBStorage**: Enhanced RocksDB shutdown process
- **StatisticsManager**: Improved statistics cleanup
- **ClientHousekeepingService**: Better client cleanup
**Impact:**
- Prevents resource leaks during broker shutdown
- Ensures graceful termination of all components
- Improves system stability and reliability
- Fixes test failures related to improper shutdown handling
### How Did You Test This Change?
<!-- In order to ensure the code quality of Apache RocketMQ, we expect every pull request to have undergone thorough testing. -->
**Manual Testing Scenarios:**
- Broker startup and shutdown cycles
- Resource cleanup verification
- State management validation
- Error handling verification
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ISSUE #9699] Optimize shutdown process and resource management [rocketmq]
Posted by "guyinyou (via GitHub)" <gi...@apache.org>.
guyinyou commented on code in PR #9700:
URL: https://github.com/apache/rocketmq/pull/9700#discussion_r2347732794
##########
broker/src/main/java/org/apache/rocketmq/broker/processor/PopBufferMergeService.java:
##########
@@ -126,9 +126,9 @@ public void run() {
if (!isShouldRunning()) {
return;
}
- while (this.buffer.size() > 0 || getOffsetTotalSize() > 0) {
- scan();
- }
+// while (this.buffer.size() > 0 || getOffsetTotalSize() > 0) {
Review Comment:
这里是不是直接删除，而不是注释
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ISSUE #9699] Optimize shutdown process and resource management [rocketmq]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #9700:
URL: https://github.com/apache/rocketmq/pull/9700#issuecomment-3289487505
## [Codecov](https://app.codecov.io/gh/apache/rocketmq/pull/9700?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `47.05882%` with `108 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 48.08%. Comparing base ([`4ae0294`](https://app.codecov.io/gh/apache/rocketmq/commit/4ae02942e58a519829a4b50c6a4d50e6c631cab2?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`b8c7185`](https://app.codecov.io/gh/apache/rocketmq/commit/b8c718592ec5fa74ea292a7b7c34e5065385ff90?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/rocketmq/pull/9700?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...apache/rocketmq/store/timer/TimerMessageStore.java](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&filepath=store%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fstore%2Ftimer%2FTimerMessageStore.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL3RpbWVyL1RpbWVyTWVzc2FnZVN0b3JlLmphdmE=) | 47.45% | [11 Missing and 20 partials :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...org/apache/rocketmq/store/DefaultMessageStore.java](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&filepath=store%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fstore%2FDefaultMessageStore.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL0RlZmF1bHRNZXNzYWdlU3RvcmUuamF2YQ==) | 51.72% | [4 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...rocketmq/common/config/AbstractRocksDBStorage.java](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&filepath=common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fcommon%2Fconfig%2FAbstractRocksDBStorage.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vY29uZmlnL0Fic3RyYWN0Um9ja3NEQlN0b3JhZ2UuamF2YQ==) | 0.00% | [12 Missing :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...main/java/org/apache/rocketmq/store/CommitLog.java](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&filepath=store%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fstore%2FCommitLog.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL0NvbW1pdExvZy5qYXZh) | 40.00% | [3 Missing and 3 partials :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...g/apache/rocketmq/store/timer/TimerCheckpoint.java](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&filepath=store%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fstore%2Ftimer%2FTimerCheckpoint.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL3RpbWVyL1RpbWVyQ2hlY2twb2ludC5qYXZh) | 44.44% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...a/org/apache/rocketmq/broker/BrokerController.java](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&filepath=broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fbroker%2FBrokerController.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvQnJva2VyQ29udHJvbGxlci5qYXZh) | 50.00% | [0 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...rocketmq/broker/processor/PopMessageProcessor.java](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&filepath=broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fbroker%2Fprocessor%2FPopMessageProcessor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvcHJvY2Vzc29yL1BvcE1lc3NhZ2VQcm9jZXNzb3IuamF2YQ==) | 0.00% | [4 Missing :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ava/org/apache/rocketmq/store/StoreCheckpoint.java](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&filepath=store%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fstore%2FStoreCheckpoint.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL1N0b3JlQ2hlY2twb2ludC5qYXZh) | 63.63% | [4 Missing :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ache/rocketmq/store/logfile/DefaultMappedFile.java](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&filepath=store%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fstore%2Flogfile%2FDefaultMappedFile.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL2xvZ2ZpbGUvRGVmYXVsdE1hcHBlZEZpbGUuamF2YQ==) | 66.66% | [4 Missing :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...rocketmq/store/queue/RocksDBConsumeQueueStore.java](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&filepath=store%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fstore%2Fqueue%2FRocksDBConsumeQueueStore.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL3F1ZXVlL1JvY2tzREJDb25zdW1lUXVldWVTdG9yZS5qYXZh) | 42.85% | [1 Missing and 3 partials :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| ... and [8 more](https://app.codecov.io/gh/apache/rocketmq/pull/9700?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## develop #9700 +/- ##
=============================================
- Coverage 48.19% 48.08% -0.11%
+ Complexity 12134 12114 -20
=============================================
Files 1313 1313
Lines 93081 93203 +122
Branches 11890 11934 +44
=============================================
- Hits 44857 44816 -41
- Misses 42697 42816 +119
- Partials 5527 5571 +44
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/rocketmq/pull/9700?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [ISSUE #9699] Optimize shutdown process and resource management [rocketmq]
Posted by "ltamber (via GitHub)" <gi...@apache.org>.
ltamber merged PR #9700:
URL: https://github.com/apache/rocketmq/pull/9700
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org