You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2022/07/05 07:43:51 UTC
[GitHub] [rocketmq] GenerousMan opened a new pull request, #4558: [RIP-43] Support Timing Messages with Arbitrary Time Delay
GenerousMan opened a new pull request, #4558:
URL: https://github.com/apache/rocketmq/pull/4558
**Make sure set the target branch to `develop`**
## What is the purpose of the change
Currently, RocketMQ's delay message feature only supports delayed delivery for specific time levels. Such delay message feature(only support specific levels of delay time) is no longer enough to support the flexible usage of rocketmq. Therefore, we need a delay message feature that supports arbitrary delay time.
I have written my proposal on the links below：
Google Doc: https://docs.google.com/document/d/1D6XWwY39p531c2aVi5HQll9iwzTUNT1haUFHqMoRkT0/edit?usp=sharing
Shimo:https://shimo.im/docs/gXqme9PKKpIeD7qo/
## Brief changelog
1. Add classes to support the feature, such as **TimerMessageStore**, **TimerWheel**, **TimerLog**, and **Slot** , etc.
2. Modify the logic of CommitLog.AsyncPutMessage(). Add two judgement(_isRolledTimerMessage(msg) / checkIfTimerMessage(msg)_) for timing messages. If the message is timing message, its topic and related properties will be converted.
3. Modify the logic of BrokerController.start()/shutdown()/initialize(), etc. Add the **TimerMessageStore** into these processes.
4. Add some functions in related files(DefaultMessageStore, CommitLog, MappedFileQueue, MessageConst, etc).
5. Add unit tests for the feature.
Follow this checklist to help us incorporate your contribution quickly and easily. Notice, `it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR`.
- [✅] Make sure there is a [Github issue](https://github.com/apache/rocketmq/issues) filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
- [✅] Format the pull request title like `[ISSUE #123] Fix UnknownException when host config not exist`. Each commit in the pull request should have a meaningful subject line and body.
- [✅] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
- [✅] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in [test module](https://github.com/apache/rocketmq/tree/master/test).
- [✅] Run `mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle` to make sure basic checks pass. Run `mvn clean install -DskipITs` to make sure unit-test pass. Run `mvn clean test-compile failsafe:integration-test` to make sure integration-test pass.
- [ ] If this contribution is large, please file an [Apache Individual Contributor License Agreement](http://www.apache.org/licenses/#clas).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] GenerousMan closed pull request #4558: [RIP-43] Support Timing Messages with Arbitrary Time Delay
Posted by GitBox <gi...@apache.org>.
GenerousMan closed pull request #4558: [RIP-43] Support Timing Messages with Arbitrary Time Delay
URL: https://github.com/apache/rocketmq/pull/4558
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] codecov-commenter commented on pull request #4558: [RIP-43] Support Timing Messages with Arbitrary Time Delay
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #4558:
URL: https://github.com/apache/rocketmq/pull/4558#issuecomment-1175755308
# [Codecov](https://codecov.io/gh/apache/rocketmq/pull/4558?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#4558](https://codecov.io/gh/apache/rocketmq/pull/4558?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (714de14) into [develop](https://codecov.io/gh/apache/rocketmq/commit/32733e76747f3db761ace75cfed6d6fdcdaee81a?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (32733e7) will **increase** coverage by `0.24%`.
> The diff coverage is `61.87%`.
```diff
@@ Coverage Diff @@
## develop #4558 +/- ##
=============================================
+ Coverage 48.16% 48.41% +0.24%
- Complexity 5127 5344 +217
=============================================
Files 649 656 +7
Lines 43021 44578 +1557
Branches 5625 5902 +277
=============================================
+ Hits 20720 21581 +861
- Misses 19798 20361 +563
- Partials 2503 2636 +133
```
| [Impacted Files](https://codecov.io/gh/apache/rocketmq/pull/4558?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...c/main/java/org/apache/rocketmq/common/MixAll.java](https://codecov.io/gh/apache/rocketmq/pull/4558/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vTWl4QWxsLmphdmE=) | `44.03% <ø> (ø)` | |
| [...va/org/apache/rocketmq/common/message/Message.java](https://codecov.io/gh/apache/rocketmq/pull/4558/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jb21tb24vbWVzc2FnZS9NZXNzYWdlLmphdmE=) | `54.21% <0.00%> (-2.75%)` | :arrow_down: |
| [...n/java/org/apache/rocketmq/store/MessageStore.java](https://codecov.io/gh/apache/rocketmq/pull/4558/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL01lc3NhZ2VTdG9yZS5qYXZh) | `0.00% <ø> (ø)` | |
| [...ain/java/org/apache/rocketmq/store/MappedFile.java](https://codecov.io/gh/apache/rocketmq/pull/4558/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL01hcHBlZEZpbGUuamF2YQ==) | `50.00% <21.42%> (-1.41%)` | :arrow_down: |
| [...a/org/apache/rocketmq/broker/BrokerController.java](https://codecov.io/gh/apache/rocketmq/pull/4558/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvQnJva2VyQ29udHJvbGxlci5qYXZh) | `46.29% <54.54%> (+0.15%)` | :arrow_up: |
| [...apache/rocketmq/store/timer/TimerMessageStore.java](https://codecov.io/gh/apache/rocketmq/pull/4558/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL3RpbWVyL1RpbWVyTWVzc2FnZVN0b3JlLmphdmE=) | `55.17% <55.17%> (ø)` | |
| [...main/java/org/apache/rocketmq/store/CommitLog.java](https://codecov.io/gh/apache/rocketmq/pull/4558/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL0NvbW1pdExvZy5qYXZh) | `75.96% <64.40%> (+0.70%)` | :arrow_up: |
| [...ava/org/apache/rocketmq/store/MappedFileQueue.java](https://codecov.io/gh/apache/rocketmq/pull/4558/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL01hcHBlZEZpbGVRdWV1ZS5qYXZh) | `63.03% <64.70%> (+0.18%)` | :arrow_up: |
| [...java/org/apache/rocketmq/store/timer/TimerLog.java](https://codecov.io/gh/apache/rocketmq/pull/4558/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL3RpbWVyL1RpbWVyTG9nLmphdmE=) | `65.11% <65.11%> (ø)` | |
| [...ache/rocketmq/store/config/MessageStoreConfig.java](https://codecov.io/gh/apache/rocketmq/pull/4558/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL2NvbmZpZy9NZXNzYWdlU3RvcmVDb25maWcuamF2YQ==) | `63.22% <65.75%> (+1.49%)` | :arrow_up: |
| ... and [55 more](https://codecov.io/gh/apache/rocketmq/pull/4558/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/rocketmq/pull/4558?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/rocketmq/pull/4558?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [32733e7...714de14](https://codecov.io/gh/apache/rocketmq/pull/4558?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] HScarb commented on a diff in pull request #4558: [RIP-43] Support Timing Messages with Arbitrary Time Delay
Posted by GitBox <gi...@apache.org>.
HScarb commented on code in PR #4558:
URL: https://github.com/apache/rocketmq/pull/4558#discussion_r913878991
##########
store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java:
##########
@@ -129,7 +131,7 @@ public class DefaultMessageStore implements MessageStore {
Executors.newSingleThreadScheduledExecutor(new ThreadFactoryImpl("DiskCheckScheduledThread"));
public DefaultMessageStore(final MessageStoreConfig messageStoreConfig, final BrokerStatsManager brokerStatsManager,
- final MessageArrivingListener messageArrivingListener, final BrokerConfig brokerConfig) throws IOException {
+ final MessageArrivingListener messageArrivingListener, final BrokerConfig brokerConfig) throws IOException {
Review Comment:
The indent of this file is incorrect.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] hzh0425 commented on pull request #4558: [RIP-43] Support Timing Messages with Arbitrary Time Delay
Posted by GitBox <gi...@apache.org>.
hzh0425 commented on PR #4558:
URL: https://github.com/apache/rocketmq/pull/4558#issuecomment-1175195392
Good job!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] coveralls commented on pull request #4558: [RIP-43] Support Timing Messages with Arbitrary Time Delay
Posted by GitBox <gi...@apache.org>.
coveralls commented on PR #4558:
URL: https://github.com/apache/rocketmq/pull/4558#issuecomment-1175756686
[](https://coveralls.io/builds/50621278)
Coverage increased (+0.4%) to 52.582% when pulling **714de144341499c5f833df66d0e6da63df387d52 on GenerousMan:AddTimer** into **ac89db8c9e06dcdcaecde0cbde80f8533577f9e0 on apache:develop**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] GenerousMan commented on a diff in pull request #4558: [RIP-43] Support Timing Messages with Arbitrary Time Delay
Posted by GitBox <gi...@apache.org>.
GenerousMan commented on code in PR #4558:
URL: https://github.com/apache/rocketmq/pull/4558#discussion_r914385190
##########
store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java:
##########
@@ -129,7 +131,7 @@ public class DefaultMessageStore implements MessageStore {
Executors.newSingleThreadScheduledExecutor(new ThreadFactoryImpl("DiskCheckScheduledThread"));
public DefaultMessageStore(final MessageStoreConfig messageStoreConfig, final BrokerStatsManager brokerStatsManager,
- final MessageArrivingListener messageArrivingListener, final BrokerConfig brokerConfig) throws IOException {
+ final MessageArrivingListener messageArrivingListener, final BrokerConfig brokerConfig) throws IOException {
Review Comment:
thx, it has been corrected in the new commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] GenerousMan commented on pull request #4558: [RIP-43] Support Timing Messages with Arbitrary Time Delay
Posted by GitBox <gi...@apache.org>.
GenerousMan commented on PR #4558:
URL: https://github.com/apache/rocketmq/pull/4558#issuecomment-1175694077
> Good job! BTW can you offer some performance test result?
@HScarb Thanks, testing will begin in the near future, and performance results will be provided at that time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org