You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2021/06/01 15:21:19 UTC
[GitHub] [rocketmq] areyouok opened a new pull request #2961: [ISSUE #2883] [Part F] Improve produce performance in M/S mode.
areyouok opened a new pull request #2961:
URL: https://github.com/apache/rocketmq/pull/2961
1. Change log level to debug: "Half offset {} has been committed/rolled back"
2. Optimise lock in WaitNotifyObject
3. Remove lock in HAService
4. Remove lock in GroupCommitService
5. Eliminate array copy in HA
6. Remove putMessage/putMessages method in CommitLog which has too many duplicated code.
7. Change default value of some parameters: sendMessageThreadPoolNums/useReentrantLockWhenPutMessage/flushCommitLogTimed/endTransactionThreadPoolNums
8. Optimise performance of asyncPutMessage (extract some code out of putMessage lock)
9. extract generation of msgId out of lock in CommitLog (now only for single message processor)
10. extract generation of topicQueueTable key out of sync code
11. extract generation of msgId out of lock in CommitLog (for batch)
12. fix ipv6 problem introduced in commit "Optimise performance of asyncPutMessage (extract some code out of putMessage lock)"
13. **Remove an duplicate MessageDecoder.string2messageProperties for each message, and prevent store "WAIT=true" property (in most case) to save 9 bytes for each message.**
14. **Improve performance of string2messageProperties/messageProperties2String, and save 1 byte for each message.**
```
Benchmark Mode Cnt Score Error Units(10000 loop in each op)
TempTest.messageProperties2String thrpt 2 2257.276 ops/s
TempTest.messageProperties2String_old thrpt 2 1464.342 ops/s
TempTest.string2messageProperties thrpt 2 1590.499 ops/s
TempTest.string2messageProperties_old thrpt 2 605.118 ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] codecov-commenter commented on pull request #2961: [ISSUE #2883] [Part F] Improve produce performance in M/S mode.
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #2961:
URL: https://github.com/apache/rocketmq/pull/2961#issuecomment-889878083
# [Codecov](https://codecov.io/gh/apache/rocketmq/pull/2961?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#2961](https://codecov.io/gh/apache/rocketmq/pull/2961?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a23df96) into [develop](https://codecov.io/gh/apache/rocketmq/commit/5e99cdbeb7d56a46059ca83923a12b9a0d80cece?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5e99cdb) will **decrease** coverage by `0.16%`.
> The diff coverage is `70.27%`.
[](https://codecov.io/gh/apache/rocketmq/pull/2961?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## develop #2961 +/- ##
=============================================
- Coverage 47.98% 47.82% -0.17%
+ Complexity 4567 4543 -24
=============================================
Files 552 552
Lines 36628 36465 -163
Branches 4844 4811 -33
=============================================
- Hits 17577 17439 -138
+ Misses 16831 16819 -12
+ Partials 2220 2207 -13
```
| [Impacted Files](https://codecov.io/gh/apache/rocketmq/pull/2961?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ocketmq/broker/processor/SendMessageProcessor.java](https://codecov.io/gh/apache/rocketmq/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvcHJvY2Vzc29yL1NlbmRNZXNzYWdlUHJvY2Vzc29yLmphdmE=) | `40.26% <37.50%> (-0.92%)` | :arrow_down: |
| [...apache/rocketmq/common/message/MessageDecoder.java](https://codecov.io/gh/apache/rocketmq/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vbWVzc2FnZS9NZXNzYWdlRGVjb2Rlci5qYXZh) | `78.14% <79.31%> (+0.09%)` | :arrow_up: |
| [.../broker/subscription/SubscriptionGroupManager.java](https://codecov.io/gh/apache/rocketmq/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvc3Vic2NyaXB0aW9uL1N1YnNjcmlwdGlvbkdyb3VwTWFuYWdlci5qYXZh) | `54.11% <0.00%> (-16.48%)` | :arrow_down: |
| [...org/apache/rocketmq/common/stats/StatsItemSet.java](https://codecov.io/gh/apache/rocketmq/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vc3RhdHMvU3RhdHNJdGVtU2V0LmphdmE=) | `41.79% <0.00%> (-10.45%)` | :arrow_down: |
| [...rocketmq/broker/pagecache/ManyMessageTransfer.java](https://codecov.io/gh/apache/rocketmq/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvcGFnZWNhY2hlL01hbnlNZXNzYWdlVHJhbnNmZXIuamF2YQ==) | `36.36% <0.00%> (-9.80%)` | :arrow_down: |
| [.../rocketmq/broker/pagecache/OneMessageTransfer.java](https://codecov.io/gh/apache/rocketmq/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvcGFnZWNhY2hlL09uZU1lc3NhZ2VUcmFuc2Zlci5qYXZh) | `24.00% <0.00%> (-9.34%)` | :arrow_down: |
| [...a/org/apache/rocketmq/logging/inner/SysLogger.java](https://codecov.io/gh/apache/rocketmq/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bG9nZ2luZy9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcm9ja2V0bXEvbG9nZ2luZy9pbm5lci9TeXNMb2dnZXIuamF2YQ==) | `28.57% <0.00%> (-5.72%)` | :arrow_down: |
| [...org/apache/rocketmq/store/ha/WaitNotifyObject.java](https://codecov.io/gh/apache/rocketmq/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL2hhL1dhaXROb3RpZnlPYmplY3QuamF2YQ==) | `66.07% <0.00%> (-5.09%)` | :arrow_down: |
| [...ketmq/client/impl/consumer/PullMessageService.java](https://codecov.io/gh/apache/rocketmq/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jbGllbnQvaW1wbC9jb25zdW1lci9QdWxsTWVzc2FnZVNlcnZpY2UuamF2YQ==) | `75.55% <0.00%> (-4.45%)` | :arrow_down: |
| [...ache/rocketmq/common/stats/MomentStatsItemSet.java](https://codecov.io/gh/apache/rocketmq/pull/2961/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vc3RhdHMvTW9tZW50U3RhdHNJdGVtU2V0LmphdmE=) | `39.13% <0.00%> (-4.35%)` | :arrow_down: |
| ... and [41 more](https://codecov.io/gh/apache/rocketmq/pull/2961/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/rocketmq/pull/2961?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/rocketmq/pull/2961?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [5e99cdb...a23df96](https://codecov.io/gh/apache/rocketmq/pull/2961?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] coveralls commented on pull request #2961: [ISSUE #2883] [Part F] Improve produce performance in M/S mode.
Posted by GitBox <gi...@apache.org>.
coveralls commented on pull request #2961:
URL: https://github.com/apache/rocketmq/pull/2961#issuecomment-889877497
[](https://coveralls.io/builds/41782369)
Coverage decreased (-0.009%) to 54.11% when pulling **a23df96790ee81147abfb38fe1c58cbbf8941828 on areyouok:dev_speed_F** into **1eaa38f63989fc06fa50a1aa0f162d8050982ce8 on apache:develop**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] caigy commented on a change in pull request #2961: [ISSUE #2883] [Part F] Improve produce performance in M/S mode.
Posted by GitBox <gi...@apache.org>.
caigy commented on a change in pull request #2961:
URL: https://github.com/apache/rocketmq/pull/2961#discussion_r673631974
##########
File path: common/src/main/java/org/apache/rocketmq/common/message/MessageDecoder.java
##########
@@ -409,7 +409,20 @@ public static MessageExt decode(
}
public static String messageProperties2String(Map<String, String> properties) {
- StringBuilder sb = new StringBuilder();
+ int len = 0;
Review comment:
Extract a method for calculating size
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] duhenglucky merged pull request #2961: [ISSUE #2883] [Part F] Improve produce performance in M/S mode.
Posted by GitBox <gi...@apache.org>.
duhenglucky merged pull request #2961:
URL: https://github.com/apache/rocketmq/pull/2961
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org