You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "GenerousMan (via GitHub)" <gi...@apache.org> on 2023/05/10 03:06:14 UTC
[GitHub] [rocketmq] GenerousMan opened a new pull request, #6729: [ISSUE #6728] Compute the confirmOffset without considering new connections
GenerousMan opened a new pull request, #6729:
URL: https://github.com/apache/rocketmq/pull/6729
[ISSUE #6728] Compute the confirmOffset without considering new connections.
<!-- Please make sure the target branch is right. In most case, the target branch should be `develop`. -->
### Which Issue(s) This PR Fixes
<!-- Please ensure that the related issue has already been created, and [link this pull request to that issue using keywords](<https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword>) to ensure automatic closure. -->
Fixes #6728
### Brief Description
1. When calculate the newConfirmOffset, skip those connections with -1 ackOffset value.
2. Add a new method, getConfirmOffsetDirectly(). This method will only return the confirmOffset without checking and re-computing. Because in some cases (If set confirmOffset wrongly) endless calling will occur between the getConfirmOffset() and computeConfirmOffset().
### How Did You Test This Change?
<!-- In order to ensure the code quality of Apache RocketMQ, we expect every pull request to have undergone thorough testing. -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] codecov-commenter commented on pull request #6729: [ISSUE #6728] Compute the confirmOffset without considering new connections
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #6729:
URL: https://github.com/apache/rocketmq/pull/6729#issuecomment-1541460238
## [Codecov](https://app.codecov.io/gh/apache/rocketmq/pull/6729?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#6729](https://app.codecov.io/gh/apache/rocketmq/pull/6729?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (52049b0) into [develop](https://app.codecov.io/gh/apache/rocketmq/commit/09ad52b9e1f36e4b153086e0de2f2c59258516b6?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (09ad52b) will **decrease** coverage by `0.05%`.
> The diff coverage is `33.33%`.
```diff
@@ Coverage Diff @@
## develop #6729 +/- ##
=============================================
- Coverage 42.88% 42.83% -0.05%
+ Complexity 9009 8995 -14
=============================================
Files 1113 1113
Lines 78587 78601 +14
Branches 10221 10224 +3
=============================================
- Hits 33700 33667 -33
- Misses 40669 40711 +42
- Partials 4218 4223 +5
```
| [Impacted Files](https://app.codecov.io/gh/apache/rocketmq/pull/6729?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...org/apache/rocketmq/store/DefaultMessageStore.java](https://app.codecov.io/gh/apache/rocketmq/pull/6729?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL0RlZmF1bHRNZXNzYWdlU3RvcmUuamF2YQ==) | `47.02% <0.00%> (-0.03%)` | :arrow_down: |
| [...main/java/org/apache/rocketmq/store/CommitLog.java](https://app.codecov.io/gh/apache/rocketmq/pull/6729?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL0NvbW1pdExvZy5qYXZh) | `64.01% <33.33%> (-0.48%)` | :arrow_down: |
| [...ketmq/store/ha/autoswitch/AutoSwitchHAService.java](https://app.codecov.io/gh/apache/rocketmq/pull/6729?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL2hhL2F1dG9zd2l0Y2gvQXV0b1N3aXRjaEhBU2VydmljZS5qYXZh) | `51.61% <50.00%> (ø)` | |
... and [32 files with indirect coverage changes](https://app.codecov.io/gh/apache/rocketmq/pull/6729/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] RongtongJin merged pull request #6729: [ISSUE #6728] Compute the confirmOffset without considering new connections
Posted by "RongtongJin (via GitHub)" <gi...@apache.org>.
RongtongJin merged PR #6729:
URL: https://github.com/apache/rocketmq/pull/6729
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] GenerousMan commented on a diff in pull request #6729: [ISSUE #6728] Compute the confirmOffset without considering new connections
Posted by "GenerousMan (via GitHub)" <gi...@apache.org>.
GenerousMan commented on code in PR #6729:
URL: https://github.com/apache/rocketmq/pull/6729#discussion_r1189421737
##########
store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/AutoSwitchHAService.java:
##########
@@ -421,14 +421,21 @@ public long computeConfirmOffset() {
for (Long syncId : currentSyncStateSet) {
if (!idList.contains(syncId) && this.brokerControllerId != null && !Objects.equals(syncId, this.brokerControllerId)) {
LOGGER.warn("Slave {} is still in syncStateSet, but has lost its connection. So new offset can't be compute.", syncId);
- return this.defaultMessageStore.getConfirmOffset();
+ // Without check and re-compute, return the confirmOffset's value directly.
+ return this.defaultMessageStore.getConfirmOffsetDirectly();
}
}
for (HAConnection connection : this.connectionList) {
final Long slaveId = ((AutoSwitchHAConnection) connection).getSlaveId();
if (currentSyncStateSet.contains(slaveId)) {
- newConfirmOffset = Math.min(newConfirmOffset, connection.getSlaveAckOffset());
+ long slaveAckOffset = connection.getSlaveAckOffset();
+ if (slaveAckOffset <= 0) {
+ // Slave's connection is just inited, the ack hasn't been calculated.
+ // So skip this ackOffset.
+ continue;
+ }
Review Comment:
Yes, I have optimized this logic as you said.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] echooymxq commented on a diff in pull request #6729: [ISSUE #6728] Compute the confirmOffset without considering new connections
Posted by "echooymxq (via GitHub)" <gi...@apache.org>.
echooymxq commented on code in PR #6729:
URL: https://github.com/apache/rocketmq/pull/6729#discussion_r1189412039
##########
store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/AutoSwitchHAService.java:
##########
@@ -421,14 +421,21 @@ public long computeConfirmOffset() {
for (Long syncId : currentSyncStateSet) {
if (!idList.contains(syncId) && this.brokerControllerId != null && !Objects.equals(syncId, this.brokerControllerId)) {
LOGGER.warn("Slave {} is still in syncStateSet, but has lost its connection. So new offset can't be compute.", syncId);
- return this.defaultMessageStore.getConfirmOffset();
+ // Without check and re-compute, return the confirmOffset's value directly.
+ return this.defaultMessageStore.getConfirmOffsetDirectly();
}
}
for (HAConnection connection : this.connectionList) {
final Long slaveId = ((AutoSwitchHAConnection) connection).getSlaveId();
if (currentSyncStateSet.contains(slaveId)) {
- newConfirmOffset = Math.min(newConfirmOffset, connection.getSlaveAckOffset());
+ long slaveAckOffset = connection.getSlaveAckOffset();
+ if (slaveAckOffset <= 0) {
+ // Slave's connection is just inited, the ack hasn't been calculated.
+ // So skip this ackOffset.
+ continue;
+ }
Review Comment:
if (currentSyncStateSet.contains(slaveId) && connection.getSlaveAckOffset() > 0) {
...
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] echooymxq commented on a diff in pull request #6729: [ISSUE #6728] Compute the confirmOffset without considering new connections
Posted by "echooymxq (via GitHub)" <gi...@apache.org>.
echooymxq commented on code in PR #6729:
URL: https://github.com/apache/rocketmq/pull/6729#discussion_r1189412039
##########
store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/AutoSwitchHAService.java:
##########
@@ -421,14 +421,21 @@ public long computeConfirmOffset() {
for (Long syncId : currentSyncStateSet) {
if (!idList.contains(syncId) && this.brokerControllerId != null && !Objects.equals(syncId, this.brokerControllerId)) {
LOGGER.warn("Slave {} is still in syncStateSet, but has lost its connection. So new offset can't be compute.", syncId);
- return this.defaultMessageStore.getConfirmOffset();
+ // Without check and re-compute, return the confirmOffset's value directly.
+ return this.defaultMessageStore.getConfirmOffsetDirectly();
}
}
for (HAConnection connection : this.connectionList) {
final Long slaveId = ((AutoSwitchHAConnection) connection).getSlaveId();
if (currentSyncStateSet.contains(slaveId)) {
- newConfirmOffset = Math.min(newConfirmOffset, connection.getSlaveAckOffset());
+ long slaveAckOffset = connection.getSlaveAckOffset();
+ if (slaveAckOffset <= 0) {
+ // Slave's connection is just inited, the ack hasn't been calculated.
+ // So skip this ackOffset.
+ continue;
+ }
Review Comment:
```java
if (currentSyncStateSet.contains(slaveId) && connection.getSlaveAckOffset() > 0) {
...
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org