You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2022/06/05 13:19:38 UTC

[GitHub] [rocketmq] hzh0425 opened a new pull request, #4413: [Summer of code] Add design and quick_start document for controller mode

hzh0425 opened a new pull request, #4413:
URL: https://github.com/apache/rocketmq/pull/4413

**Make sure set the target branch to `develop`**

## What is the purpose of the change

tracking issue: https://github.com/apache/rocketmq/issues/4330

## Brief changelog

Add design and quick_start document for controller mode

## Verifying this change

XXXX

Follow this checklist to help us incorporate your contribution quickly and easily. Notice, `it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR`.

- [x] Make sure there is a [Github issue](https://github.com/apache/rocketmq/issues) filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
- [x] Format the pull request title like `[ISSUE #123] Fix UnknownException when host config not exist`. Each commit in the pull request should have a meaningful subject line and body.
- [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
- [x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in [test module](https://github.com/apache/rocketmq/tree/master/test).
- [x] Run `mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle` to make sure basic checks pass. Run `mvn clean install -DskipITs` to make sure unit-test pass. Run `mvn clean test-compile failsafe:integration-test` to make sure integration-test pass.
- [ ] If this contribution is large, please file an [Apache Individual Contributor License Agreement](http://www.apache.org/licenses/#clas).

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] RongtongJin commented on a diff in pull request #4413: [Summer of code] Add design and quick_start document for controller mode

Posted by GitBox <gi...@apache.org>.

RongtongJin commented on code in PR #4413:
URL: https://github.com/apache/rocketmq/pull/4413#discussion_r889780199


##########
docs/cn/controller/design.md:
##########
@@ -0,0 +1,206 @@
+# 背景
+
+当前 RocketMQ Raft 模式主要是利用 DLedger Commitlog 替换原来的 Commitlog，使 Commitlog 拥有选举复制能力，但这也造成了一些问题：
+
+- Raft 模式下，Broker组内副本数必须是三副本及以上。
+- RocketMQ 存在两套 HA 复制流程，且 Raft 模式下的复制无法利用 RocketMQ 原生的存储能力。
+- Raft 模式下, 日志复制性能并不高效。
+
+因此我们希望利用 DLedger 实现一个基于 Raft 的一致性模块（DLedger Controller），并当作一个可选的选主组件, 支持独立部署, 也可以嵌入在 Nameserver 中，Broker 通过与 Controller 的交互完成 Master 的选举, 从而解决上述问题, 我们奖该新模式称为 Controller 模式。

Review Comment:
   我们将该新模式称为 Controller 模式



##########
broker/src/main/java/org/apache/rocketmq/broker/hacontroller/ReplicasManager.java:
##########
@@ -206,7 +206,10 @@ public void changeToSlave(final String newMasterAddress, final int newMasterEpoc
                 handleSlaveSynchronize(BrokerRole.SLAVE);
 
                 // Notify ha service, change to slave
-                this.haService.changeToSlave(newMasterAddress, newMasterEpoch, this.brokerConfig.getBrokerId());
+                if (!this.haService.changeToSlave(newMasterAddress, newMasterEpoch, this.brokerConfig.getBrokerId())) {
+                    LOGGER.info("Failed to change ha role to slave");
+                    return;
+                }

Review Comment:
   如果失败了的话，上面的工作要不要回滚？



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] hzh0425 closed pull request #4413: [Summer of code] Add design and quick_start document for controller mode

Posted by GitBox <gi...@apache.org>.

hzh0425 closed pull request #4413: [Summer of code] Add design and quick_start document for controller mode
URL: https://github.com/apache/rocketmq/pull/4413


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] coveralls commented on pull request #4413: [Summer of code] Add design and quick_start document for controller mode

Posted by GitBox <gi...@apache.org>.

coveralls commented on PR #4413:
URL: https://github.com/apache/rocketmq/pull/4413#issuecomment-1146814138

   
   [![Coverage Status](https://coveralls.io/builds/49738803/badge)](https://coveralls.io/builds/49738803)
   
   Coverage increased (+0.1%) to 47.486% when pulling **1355e2590b36463940e7ab964f51c0927f55ff1b on hzh0425:feature/add-document** into **8515a8d0915d1d2e2e41a0fa3ea32ba6efc1595f on apache:5.0.0-beta-dledger-controller**.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] RongtongJin merged pull request #4413: [Summer of code] Add design and quick_start document for controller mode

Posted by GitBox <gi...@apache.org>.

RongtongJin merged PR #4413:
URL: https://github.com/apache/rocketmq/pull/4413


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] hzh0425 commented on pull request #4413: [Summer of code] Add design and quick_start document for controller mode

Posted by GitBox <gi...@apache.org>.

hzh0425 commented on PR #4413:
URL: https://github.com/apache/rocketmq/pull/4413#issuecomment-1146803897

   @RongtongJin please help review the documents (Design and quick start), and after the review, I will add the English version of the documents.
    Also, I will add a deploy_guide document.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] codecov-commenter commented on pull request #4413: [Summer of code] Add design and quick_start document for controller mode

Posted by GitBox <gi...@apache.org>.

codecov-commenter commented on PR #4413:
URL: https://github.com/apache/rocketmq/pull/4413#issuecomment-1146842299

   # [Codecov](https://codecov.io/gh/apache/rocketmq/pull/4413?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#4413](https://codecov.io/gh/apache/rocketmq/pull/4413?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f76adde) into [5.0.0-beta-dledger-controller](https://codecov.io/gh/apache/rocketmq/commit/8515a8d0915d1d2e2e41a0fa3ea32ba6efc1595f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8515a8d) will **decrease** coverage by `0.04%`.
   > The diff coverage is `0.00%`.
   
   ```diff
   @@                         Coverage Diff                         @@
   ##             5.0.0-beta-dledger-controller    #4413      +/-   ##
   ===================================================================
   - Coverage                            43.27%   43.23%   -0.05%     
   + Complexity                            6340     6331       -9     
   ===================================================================
     Files                                  840      840              
     Lines                                59699    59702       +3     
     Branches                              8138     8139       +1     
   ===================================================================
   - Hits                                 25833    25810      -23     
   - Misses                               30515    30539      +24     
   - Partials                              3351     3353       +2     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/rocketmq/pull/4413?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [.../rocketmq/broker/hacontroller/ReplicasManager.java](https://codecov.io/gh/apache/rocketmq/pull/4413/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvaGFjb250cm9sbGVyL1JlcGxpY2FzTWFuYWdlci5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...ketmq/store/ha/autoswitch/AutoSwitchHAService.java](https://codecov.io/gh/apache/rocketmq/pull/4413/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL2hhL2F1dG9zd2l0Y2gvQXV0b1N3aXRjaEhBU2VydmljZS5qYXZh) | `55.31% <0.00%> (ø)` | |
   | [...q/namesrv/routeinfo/BrokerHousekeepingService.java](https://codecov.io/gh/apache/rocketmq/pull/4413/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bmFtZXNydi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcm9ja2V0bXEvbmFtZXNydi9yb3V0ZWluZm8vQnJva2VySG91c2VrZWVwaW5nU2VydmljZS5qYXZh) | `72.72% <0.00%> (-9.10%)` | :arrow_down: |
   | [...ava/org/apache/rocketmq/filter/util/BitsArray.java](https://codecov.io/gh/apache/rocketmq/pull/4413/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZmlsdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9maWx0ZXIvdXRpbC9CaXRzQXJyYXkuamF2YQ==) | `59.82% <0.00%> (-2.57%)` | :arrow_down: |
   | [...he/rocketmq/client/trace/AsyncTraceDispatcher.java](https://codecov.io/gh/apache/rocketmq/pull/4413/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9jbGllbnQvdHJhY2UvQXN5bmNUcmFjZURpc3BhdGNoZXIuamF2YQ==) | `77.22% <0.00%> (-1.49%)` | :arrow_down: |
   | [.../apache/rocketmq/logging/inner/LoggingBuilder.java](https://codecov.io/gh/apache/rocketmq/pull/4413/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bG9nZ2luZy9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcm9ja2V0bXEvbG9nZ2luZy9pbm5lci9Mb2dnaW5nQnVpbGRlci5qYXZh) | `63.44% <0.00%> (-1.27%)` | :arrow_down: |
   | [...cketmq/broker/schedule/ScheduleMessageService.java](https://codecov.io/gh/apache/rocketmq/pull/4413/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvc2NoZWR1bGUvU2NoZWR1bGVNZXNzYWdlU2VydmljZS5qYXZh) | `56.00% <0.00%> (-1.18%)` | :arrow_down: |
   | [...che/rocketmq/namesrv/kvconfig/KVConfigManager.java](https://codecov.io/gh/apache/rocketmq/pull/4413/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bmFtZXNydi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcm9ja2V0bXEvbmFtZXNydi9rdmNvbmZpZy9LVkNvbmZpZ01hbmFnZXIuamF2YQ==) | `59.18% <0.00%> (-1.03%)` | :arrow_down: |
   | [.../org/apache/rocketmq/store/ha/DefaultHAClient.java](https://codecov.io/gh/apache/rocketmq/pull/4413/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3JvY2tldG1xL3N0b3JlL2hhL0RlZmF1bHRIQUNsaWVudC5qYXZh) | `58.46% <0.00%> (-0.52%)` | :arrow_down: |
   | [...che/rocketmq/acl/plain/PlainPermissionManager.java](https://codecov.io/gh/apache/rocketmq/pull/4413/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YWNsL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9hY2wvcGxhaW4vUGxhaW5QZXJtaXNzaW9uTWFuYWdlci5qYXZh) | `75.19% <0.00%> (-0.52%)` | :arrow_down: |
   | ... and [3 more](https://codecov.io/gh/apache/rocketmq/pull/4413/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/rocketmq/pull/4413?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/rocketmq/pull/4413?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [8515a8d...f76adde](https://codecov.io/gh/apache/rocketmq/pull/4413?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org