You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by GitBox <gi...@apache.org> on 2023/01/06 02:59:05 UTC
[GitHub] [rocketmq] Oliverwqcwrw opened a new pull request, #5834: [ISSUE #5832] Fix consumerCount increasing rapidly without sending message
Oliverwqcwrw opened a new pull request, #5834:
URL: https://github.com/apache/rocketmq/pull/5834
**Make sure set the target branch to `develop`**
## What is the purpose of the change
Close #5832
## Brief changelog
Fix consumerCount increasing rapidly without sending message
## Verifying this change
XXXX
Follow this checklist to help us incorporate your contribution quickly and easily. Notice, `it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR`.
- [x] Make sure there is a [Github issue](https://github.com/apache/rocketmq/issues) filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
- [x] Format the pull request title like `[ISSUE #123] Fix UnknownException when host config not exist`. Each commit in the pull request should have a meaningful subject line and body.
- [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
- [x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in [test module](https://github.com/apache/rocketmq/tree/master/test).
- [x] Run `mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle` to make sure basic checks pass. Run `mvn clean install -DskipITs` to make sure unit-test pass. Run `mvn clean test-compile failsafe:integration-test` to make sure integration-test pass.
- [ ] If this contribution is large, please file an [Apache Individual Contributor License Agreement](http://www.apache.org/licenses/#clas).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Oliverwqcwrw commented on a diff in pull request #5834: [ISSUE #5832] Fix consumerCount increasing rapidly without sending message
Posted by GitBox <gi...@apache.org>.
Oliverwqcwrw commented on code in PR #5834:
URL: https://github.com/apache/rocketmq/pull/5834#discussion_r1063162445
##########
store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java:
##########
@@ -849,7 +849,11 @@ public GetMessageResult getMessage(final String group, final String topic, final
continue;
}
- this.storeStatsService.getGetMessageTransferredMsgCount().add(cqUnit.getBatchNum());
+ if (topicConfig.isPresent()) {
+ if (!topicConfig.get().getTopicName().contains(TopicValidator.RMQ_SYS_REVIVE_LOG)) {
Review Comment:
IMO, It will broke the current structure of StoreStatsService if moving the getMessageTranferredMsgCount to BrokerStatsManager,
because not only is getMessageTransferredMsgCount in there but also other metrics is there,
whether moving the other metrics if moving getMessageTransferredMsgCount to BrokerStatsManager?
BTW,other system topics can't influence the consume num
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Oliverwqcwrw merged pull request #5834: [ISSUE #5832] Fix consumerCount increasing rapidly without sending message
Posted by GitBox <gi...@apache.org>.
Oliverwqcwrw merged PR #5834:
URL: https://github.com/apache/rocketmq/pull/5834
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Oliverwqcwrw commented on a diff in pull request #5834: [ISSUE #5832] Fix consumerCount increasing rapidly without sending message
Posted by GitBox <gi...@apache.org>.
Oliverwqcwrw commented on code in PR #5834:
URL: https://github.com/apache/rocketmq/pull/5834#discussion_r1064069946
##########
store/src/main/java/org/apache/rocketmq/store/stats/BrokerStats.java:
##########
@@ -88,6 +88,6 @@ public long getMsgPutTotalTodayNow() {
}
public long getMsgGetTotalTodayNow() {
Review Comment:
The getMsgGetTotalYesterdayMorning's data is from getGetMessageTransferredMsgCount,
so it's ok to replace the getGetMessageTransferredMsgCount to getNumsWithoutSystemTopic without adding other counter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Oliverwqcwrw commented on a diff in pull request #5834: [ISSUE #5832] Fix consumerCount increasing rapidly without sending message
Posted by GitBox <gi...@apache.org>.
Oliverwqcwrw commented on code in PR #5834:
URL: https://github.com/apache/rocketmq/pull/5834#discussion_r1063213642
##########
store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java:
##########
@@ -849,7 +849,11 @@ public GetMessageResult getMessage(final String group, final String topic, final
continue;
}
- this.storeStatsService.getGetMessageTransferredMsgCount().add(cqUnit.getBatchNum());
+ if (topicConfig.isPresent()) {
+ if (!topicConfig.get().getTopicName().contains(TopicValidator.RMQ_SYS_REVIVE_LOG)) {
Review Comment:
Get it, I will polish it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] ShadowySpirits commented on a diff in pull request #5834: [ISSUE #5832] Fix consumerCount increasing rapidly without sending message
Posted by GitBox <gi...@apache.org>.
ShadowySpirits commented on code in PR #5834:
URL: https://github.com/apache/rocketmq/pull/5834#discussion_r1063068907
##########
store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java:
##########
@@ -849,7 +849,11 @@ public GetMessageResult getMessage(final String group, final String topic, final
continue;
}
- this.storeStatsService.getGetMessageTransferredMsgCount().add(cqUnit.getBatchNum());
+ if (topicConfig.isPresent()) {
+ if (!topicConfig.get().getTopicName().contains(TopicValidator.RMQ_SYS_REVIVE_LOG)) {
Review Comment:
Store metrics should not exclude specified topics. It's better to add a new counter in BrokerStatsManager to calculate the message transfer count excluding all system topics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Oliverwqcwrw commented on a diff in pull request #5834: [ISSUE #5832] Fix consumerCount increasing rapidly without sending message
Posted by GitBox <gi...@apache.org>.
Oliverwqcwrw commented on code in PR #5834:
URL: https://github.com/apache/rocketmq/pull/5834#discussion_r1064069929
##########
store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java:
##########
@@ -848,8 +848,8 @@ public GetMessageResult getMessage(final String group, final String topic, final
selectResult.release();
continue;
}
-
this.storeStatsService.getGetMessageTransferredMsgCount().add(cqUnit.getBatchNum());
+ this.brokerStatsManager.incBrokerGetNumsWithoutSystemTopic(topic, cqUnit.getBatchNum());
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Oliverwqcwrw commented on a diff in pull request #5834: [ISSUE #5832] Fix consumerCount increasing rapidly without sending message
Posted by GitBox <gi...@apache.org>.
Oliverwqcwrw commented on code in PR #5834:
URL: https://github.com/apache/rocketmq/pull/5834#discussion_r1063162445
##########
store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java:
##########
@@ -849,7 +849,11 @@ public GetMessageResult getMessage(final String group, final String topic, final
continue;
}
- this.storeStatsService.getGetMessageTransferredMsgCount().add(cqUnit.getBatchNum());
+ if (topicConfig.isPresent()) {
+ if (!topicConfig.get().getTopicName().contains(TopicValidator.RMQ_SYS_REVIVE_LOG)) {
Review Comment:
IMO, It will broke the current structure of StoreStatsService if moving the getMessageTranferredMsgCount to BrokerStatsManager,
because not only is getMessageTransferredMsgCount in there but also other metrics is there,
whether moving the other metrics if moving getMessageTransferredMsgCount to BrokerStatsManager?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] ShadowySpirits commented on a diff in pull request #5834: [ISSUE #5832] Fix consumerCount increasing rapidly without sending message
Posted by GitBox <gi...@apache.org>.
ShadowySpirits commented on code in PR #5834:
URL: https://github.com/apache/rocketmq/pull/5834#discussion_r1063196401
##########
store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java:
##########
@@ -849,7 +849,11 @@ public GetMessageResult getMessage(final String group, final String topic, final
continue;
}
- this.storeStatsService.getGetMessageTransferredMsgCount().add(cqUnit.getBatchNum());
+ if (topicConfig.isPresent()) {
+ if (!topicConfig.get().getTopicName().contains(TopicValidator.RMQ_SYS_REVIVE_LOG)) {
Review Comment:
I mean add a new counter in BrokerStatsManager and do not remove the current counter. The new counter only records the throughput of the user's topic. This is more general and semantically clear.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] ShadowySpirits commented on a diff in pull request #5834: [ISSUE #5832] Fix consumerCount increasing rapidly without sending message
Posted by GitBox <gi...@apache.org>.
ShadowySpirits commented on code in PR #5834:
URL: https://github.com/apache/rocketmq/pull/5834#discussion_r1063998639
##########
store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java:
##########
@@ -848,8 +848,8 @@ public GetMessageResult getMessage(final String group, final String topic, final
selectResult.release();
continue;
}
-
this.storeStatsService.getGetMessageTransferredMsgCount().add(cqUnit.getBatchNum());
+ this.brokerStatsManager.incBrokerGetNumsWithoutSystemTopic(topic, cqUnit.getBatchNum());
Review Comment:
Using brokerStatsManager in the store module looks weird... It's better to add this counter in BrokerStatsManager#incBrokerGetNums.
##########
store/src/main/java/org/apache/rocketmq/store/stats/BrokerStats.java:
##########
@@ -88,6 +88,6 @@ public long getMsgPutTotalTodayNow() {
}
public long getMsgGetTotalTodayNow() {
Review Comment:
How about dealing with getMsgGetTotalYesterdayMorning in the same way?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org