You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "RongtongJin (via GitHub)" <gi...@apache.org> on 2023/04/19 03:34:19 UTC
[GitHub] [rocketmq] RongtongJin opened a new pull request, #6618: [ISSUE #6609] Fix the issue that consume queue building exceeds confirmOffset when node restarts to recover
RongtongJin opened a new pull request, #6618:
URL: https://github.com/apache/rocketmq/pull/6618
**Make sure set the target branch to `develop`**
## What is the purpose of the change
<!--
If this PR fixes a GitHub issue, please add `fixes #<XXX>` or `closes #<XXX>`. Please refer to the documentation for more information:
https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue
-->
fix #6609
## Brief changelog
Fix the issue that consume queue building exceeds confirmOffset when node restarts to recover
## Verifying this change
XXXX
Follow this checklist to help us incorporate your contribution quickly and easily. Notice, `it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR`.
- [x] Make sure there is a [Github issue](https://github.com/apache/rocketmq/issues) filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
- [x] Format the pull request title like `[ISSUE #123] Fix UnknownException when host config not exist`. Each commit in the pull request should have a meaningful subject line and body.
- [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
- [x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in [test module](https://github.com/apache/rocketmq/tree/master/test).
- [x] Run `mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle` to make sure basic checks pass. Run `mvn clean install -DskipITs` to make sure unit-test pass. Run `mvn clean test-compile failsafe:integration-test` to make sure integration-test pass.
- [ ] If this contribution is large, please file an [Apache Individual Contributor License Agreement](http://www.apache.org/licenses/#clas).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] GenerousMan commented on pull request #6618: [ISSUE #6609] Fix the issue that consume queue building exceeds confirmOffset when node restarts to recover
Posted by "GenerousMan (via GitHub)" <gi...@apache.org>.
GenerousMan commented on PR #6618:
URL: https://github.com/apache/rocketmq/pull/6618#issuecomment-1533951585
LGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] fuyou001 commented on a diff in pull request #6618: [ISSUE #6609] Fix the issue that consume queue building exceeds confirmOffset when node restarts to recover
Posted by "fuyou001 (via GitHub)" <gi...@apache.org>.
fuyou001 commented on code in PR #6618:
URL: https://github.com/apache/rocketmq/pull/6618#discussion_r1171105169
##########
store/src/main/java/org/apache/rocketmq/store/StoreCheckpoint.java:
##########
@@ -53,6 +54,7 @@ public StoreCheckpoint(final String scpPath) throws IOException {
this.logicsMsgTimestamp = this.mappedByteBuffer.getLong(8);
this.indexMsgTimestamp = this.mappedByteBuffer.getLong(16);
this.masterFlushedOffset = this.mappedByteBuffer.getLong(24);
+ this.confirmPhyOffset = this.mappedByteBuffer.getLong(32);
Review Comment:
这块兼容性考虑下，老版本没有这逻辑。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] RongtongJin commented on a diff in pull request #6618: [ISSUE #6609] Fix the issue that consume queue building exceeds confirmOffset when node restarts to recover
Posted by "RongtongJin (via GitHub)" <gi...@apache.org>.
RongtongJin commented on code in PR #6618:
URL: https://github.com/apache/rocketmq/pull/6618#discussion_r1181075568
##########
store/src/main/java/org/apache/rocketmq/store/StoreCheckpoint.java:
##########
@@ -53,6 +54,7 @@ public StoreCheckpoint(final String scpPath) throws IOException {
this.logicsMsgTimestamp = this.mappedByteBuffer.getLong(8);
this.indexMsgTimestamp = this.mappedByteBuffer.getLong(16);
this.masterFlushedOffset = this.mappedByteBuffer.getLong(24);
+ this.confirmPhyOffset = this.mappedByteBuffer.getLong(32);
Review Comment:
如果是升级上来，之前没有设置过，则默认会读取到0。
在非controller模式下，不会用到该参数，因此不会有任何影响。
在controller模式下，由于confirmOffset为0，消息会重复dispatch，此时会打印“Maybe try to build consume queue repeatedly xxx”日志，但不会真正的dispatch。
由于老版本没有持久化confirmOffset，因此没有办法避免第一次启动时重复dispatch和日志的打印，但正常启动后不会对功能有影响。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] codecov-commenter commented on pull request #6618: [ISSUE #6609] Fix the issue that consume queue building exceeds confirmOffset when node restarts to recover
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #6618:
URL: https://github.com/apache/rocketmq/pull/6618#issuecomment-1514207205
## [Codecov](https://codecov.io/gh/apache/rocketmq/pull/6618?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#6618](https://codecov.io/gh/apache/rocketmq/pull/6618?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (20e91d7) into [develop](https://codecov.io/gh/apache/rocketmq/commit/ca006ca9abfc43e317e1b1eb486927c45b2cc633?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ca006ca) will **increase** coverage by `0.02%`.
> The diff coverage is `64.15%`.
```diff
@@ Coverage Diff @@
## develop #6618 +/- ##
=============================================
+ Coverage 43.07% 43.10% +0.02%
- Complexity 8988 9002 +14
=============================================
Files 1107 1107
Lines 78287 78306 +19
Branches 10202 10209 +7
=============================================
+ Hits 33726 33757 +31
+ Misses 40335 40327 -8
+ Partials 4226 4222 -4
```
| [Impacted Files](https://codecov.io/gh/apache/rocketmq/pull/6618?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...a/org/apache/rocketmq/broker/BrokerController.java](https://codecov.io/gh/apache/rocketmq/pull/6618?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvQnJva2VyQ29udHJvbGxlci5qYXZh) | `43.53% <0.00%> (ø)` | |
| [...he/rocketmq/broker/controller/ReplicasManager.java](https://codecov.io/gh/apache/rocketmq/pull/6618?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvY29udHJvbGxlci9SZXBsaWNhc01hbmFnZXIuamF2YQ==) | `45.98% <33.33%> (+0.12%)` | :arrow_up: |
| [...main/java/org/apache/rocketmq/store/CommitLog.java](https://codecov.io/gh/apache/rocketmq/pull/6618?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL0NvbW1pdExvZy5qYXZh) | `64.90% <53.57%> (-0.24%)` | :arrow_down: |
| [...n/java/org/apache/rocketmq/store/RunningFlags.java](https://codecov.io/gh/apache/rocketmq/pull/6618?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL1J1bm5pbmdGbGFncy5qYXZh) | `39.58% <83.33%> (+6.25%)` | :arrow_up: |
| [...ava/org/apache/rocketmq/store/StoreCheckpoint.java](https://codecov.io/gh/apache/rocketmq/pull/6618?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL1N0b3JlQ2hlY2twb2ludC5qYXZh) | `87.50% <83.33%> (-0.44%)` | :arrow_down: |
| [...org/apache/rocketmq/store/DefaultMessageStore.java](https://codecov.io/gh/apache/rocketmq/pull/6618?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL0RlZmF1bHRNZXNzYWdlU3RvcmUuamF2YQ==) | `46.91% <100.00%> (+0.15%)` | :arrow_up: |
| [...cketmq/store/ha/autoswitch/AutoSwitchHAClient.java](https://codecov.io/gh/apache/rocketmq/pull/6618?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL2hhL2F1dG9zd2l0Y2gvQXV0b1N3aXRjaEhBQ2xpZW50LmphdmE=) | `75.73% <100.00%> (-0.46%)` | :arrow_down: |
| [...mq/store/ha/autoswitch/AutoSwitchHAConnection.java](https://codecov.io/gh/apache/rocketmq/pull/6618?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL2hhL2F1dG9zd2l0Y2gvQXV0b1N3aXRjaEhBQ29ubmVjdGlvbi5qYXZh) | `70.76% <100.00%> (+0.27%)` | :arrow_up: |
| [...ketmq/store/ha/autoswitch/AutoSwitchHAService.java](https://codecov.io/gh/apache/rocketmq/pull/6618?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL2hhL2F1dG9zd2l0Y2gvQXV0b1N3aXRjaEhBU2VydmljZS5qYXZh) | `52.63% <100.00%> (-0.46%)` | :arrow_down: |
... and [19 files with indirect coverage changes](https://codecov.io/gh/apache/rocketmq/pull/6618/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] ShannonDing merged pull request #6618: [ISSUE #6609] Fix the issue that consume queue building exceeds confirmOffset when node restarts to recover
Posted by "ShannonDing (via GitHub)" <gi...@apache.org>.
ShannonDing merged PR #6618:
URL: https://github.com/apache/rocketmq/pull/6618
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org