You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tubemq.apache.org by GitBox <gi...@apache.org> on 2020/12/29 02:23:53 UTC
[GitHub] [incubator-tubemq] EMsnap opened a new pull request #374: [TUBEMQ-481] add new feature - clone broker
EMsnap opened a new pull request #374:
URL: https://github.com/apache/incubator-tubemq/pull/374
[TUBEMQ-481] add new feature - clone broker
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tubemq] EMsnap commented on a change in pull request #374: [TUBEMQ-481] add new feature - clone broker
Posted by GitBox <gi...@apache.org>.
EMsnap commented on a change in pull request #374:
URL: https://github.com/apache/incubator-tubemq/pull/374#discussion_r549552283
##########
File path: tubemq-manager/src/main/java/org/apache/tubemq/manager/service/NodeService.java
##########
@@ -120,6 +123,106 @@ private TubeHttpTopicInfoList requestTopicConfigInfo(NodeEntry nodeEntry, String
}
+ public TubeMQResult cloneBrokersWithTopic(CloneBrokersReq req, int clusterId) throws Exception {
+
+ // 1. query source broker config
+ QueryBrokerCfgReq queryReq = QueryBrokerCfgReq.getReq(req.getSourceBrokerId());
+ NodeEntry masterEntry = nodeRepository.findNodeEntryByClusterIdIsAndMasterIsTrue(
+ clusterId);
+ BrokerStatusInfo brokerStatusInfo = getBrokerStatusInfo(queryReq, masterEntry);
+
+ // 2. use source broker config to clone brokers
+ BrokerConf sourceBrokerConf = brokerStatusInfo.getData().get(0);
+ AddBrokersReq addBrokersReq = getBatchAddBrokersReq(req, clusterId, sourceBrokerConf);
+
+ // 3. request master, return broker ids generated by master
+ AddBrokerResult addBrokerResult = addBrokersToClusterWithId(addBrokersReq, masterEntry);
+
+ // might have duplicate brokers
+ if (addBrokerResult.getErrCode() != 0) {
+ return TubeMQResult.getErrorResult(addBrokerResult.getErrMsg());
+ }
+ List<Integer> brokerIds = getBrokerIds(addBrokerResult);
+ List<AddTopicReq> addTopicReqs = req.getAddTopicReqs();
+
+ // 4. add topic to brokers
+ return addTopicToBrokers(masterEntry, brokerIds, addTopicReqs);
+ }
+
+ private TubeMQResult addTopicToBrokers(NodeEntry masterEntry, List<Integer> brokerIds, List<AddTopicReq> addTopicReqs) {
+ TubeMQResult tubeResult = new TubeMQResult();
+ AddTopicsResult addTopicsResult = new AddTopicsResult();
+
+ if (CollectionUtils.isEmpty(addTopicReqs)) {
+ return tubeResult;
+ }
+ addTopicReqs.forEach(addTopicReq -> {
+ try {
+ String brokerStr = StringUtils.join(brokerIds, ",");
Review comment:
roger
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tubemq] codecov-io edited a comment on pull request #374: [TUBEMQ-481] add new feature - clone broker
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #374:
URL: https://github.com/apache/incubator-tubemq/pull/374#issuecomment-751923775
# [Codecov](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=h1) Report
> Merging [#374](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=desc) (23a28ed) into [TUBEMQ-421](https://codecov.io/gh/apache/incubator-tubemq/commit/746b8c182ce0570e8c694e506c8c4f3f5a5c60e9?el=desc) (746b8c1) will **decrease** coverage by `0.00%`.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## TUBEMQ-421 #374 +/- ##
===============================================
- Coverage 8.02% 8.02% -0.01%
Complexity 466 466
===============================================
Files 249 249
Lines 27115 27125 +10
Branches 4440 4441 +1
===============================================
Hits 2176 2176
- Misses 24468 24478 +10
Partials 471 471
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...er/master/web/handler/WebBrokerDefConfHandler.java](https://codecov.io/gh/apache/incubator-tubemq/pull/374/diff?src=pr&el=tree#diff-dHViZW1xLXNlcnZlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvdHViZW1xL3NlcnZlci9tYXN0ZXIvd2ViL2hhbmRsZXIvV2ViQnJva2VyRGVmQ29uZkhhbmRsZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=footer). Last update [746b8c1...23a28ed](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tubemq] codecov-io commented on pull request #374: [TUBEMQ-481] add new feature - clone broker
Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #374:
URL: https://github.com/apache/incubator-tubemq/pull/374#issuecomment-751923775
# [Codecov](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=h1) Report
> Merging [#374](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=desc) (b021e22) into [TUBEMQ-421](https://codecov.io/gh/apache/incubator-tubemq/commit/746b8c182ce0570e8c694e506c8c4f3f5a5c60e9?el=desc) (746b8c1) will **decrease** coverage by `0.00%`.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## TUBEMQ-421 #374 +/- ##
===============================================
- Coverage 8.02% 8.02% -0.01%
Complexity 466 466
===============================================
Files 249 249
Lines 27115 27125 +10
Branches 4440 4441 +1
===============================================
Hits 2176 2176
- Misses 24468 24478 +10
Partials 471 471
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...er/master/web/handler/WebBrokerDefConfHandler.java](https://codecov.io/gh/apache/incubator-tubemq/pull/374/diff?src=pr&el=tree#diff-dHViZW1xLXNlcnZlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvdHViZW1xL3NlcnZlci9tYXN0ZXIvd2ViL2hhbmRsZXIvV2ViQnJva2VyRGVmQ29uZkhhbmRsZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=footer). Last update [746b8c1...b021e22](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tubemq] codecov-io edited a comment on pull request #374: [TUBEMQ-481] add new feature - clone broker
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #374:
URL: https://github.com/apache/incubator-tubemq/pull/374#issuecomment-751923775
# [Codecov](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=h1) Report
> Merging [#374](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=desc) (501695f) into [TUBEMQ-421](https://codecov.io/gh/apache/incubator-tubemq/commit/746b8c182ce0570e8c694e506c8c4f3f5a5c60e9?el=desc) (746b8c1) will **decrease** coverage by `0.00%`.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## TUBEMQ-421 #374 +/- ##
===============================================
- Coverage 8.02% 8.02% -0.01%
Complexity 466 466
===============================================
Files 249 249
Lines 27115 27125 +10
Branches 4440 4441 +1
===============================================
Hits 2176 2176
- Misses 24468 24478 +10
Partials 471 471
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...er/master/web/handler/WebBrokerDefConfHandler.java](https://codecov.io/gh/apache/incubator-tubemq/pull/374/diff?src=pr&el=tree#diff-dHViZW1xLXNlcnZlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvdHViZW1xL3NlcnZlci9tYXN0ZXIvd2ViL2hhbmRsZXIvV2ViQnJva2VyRGVmQ29uZkhhbmRsZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=footer). Last update [746b8c1...501695f](https://codecov.io/gh/apache/incubator-tubemq/pull/374?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tubemq] yuanboliu merged pull request #374: [TUBEMQ-481] add new feature - clone broker
Posted by GitBox <gi...@apache.org>.
yuanboliu merged pull request #374:
URL: https://github.com/apache/incubator-tubemq/pull/374
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tubemq] yuanboliu commented on a change in pull request #374: [TUBEMQ-481] add new feature - clone broker
Posted by GitBox <gi...@apache.org>.
yuanboliu commented on a change in pull request #374:
URL: https://github.com/apache/incubator-tubemq/pull/374#discussion_r549550635
##########
File path: tubemq-manager/src/main/java/org/apache/tubemq/manager/service/NodeService.java
##########
@@ -120,6 +123,106 @@ private TubeHttpTopicInfoList requestTopicConfigInfo(NodeEntry nodeEntry, String
}
+ public TubeMQResult cloneBrokersWithTopic(CloneBrokersReq req, int clusterId) throws Exception {
+
+ // 1. query source broker config
+ QueryBrokerCfgReq queryReq = QueryBrokerCfgReq.getReq(req.getSourceBrokerId());
+ NodeEntry masterEntry = nodeRepository.findNodeEntryByClusterIdIsAndMasterIsTrue(
+ clusterId);
+ BrokerStatusInfo brokerStatusInfo = getBrokerStatusInfo(queryReq, masterEntry);
+
+ // 2. use source broker config to clone brokers
+ BrokerConf sourceBrokerConf = brokerStatusInfo.getData().get(0);
+ AddBrokersReq addBrokersReq = getBatchAddBrokersReq(req, clusterId, sourceBrokerConf);
+
+ // 3. request master, return broker ids generated by master
+ AddBrokerResult addBrokerResult = addBrokersToClusterWithId(addBrokersReq, masterEntry);
+
+ // might have duplicate brokers
+ if (addBrokerResult.getErrCode() != 0) {
+ return TubeMQResult.getErrorResult(addBrokerResult.getErrMsg());
+ }
+ List<Integer> brokerIds = getBrokerIds(addBrokerResult);
+ List<AddTopicReq> addTopicReqs = req.getAddTopicReqs();
+
+ // 4. add topic to brokers
+ return addTopicToBrokers(masterEntry, brokerIds, addTopicReqs);
+ }
+
+ private TubeMQResult addTopicToBrokers(NodeEntry masterEntry, List<Integer> brokerIds, List<AddTopicReq> addTopicReqs) {
+ TubeMQResult tubeResult = new TubeMQResult();
+ AddTopicsResult addTopicsResult = new AddTopicsResult();
+
+ if (CollectionUtils.isEmpty(addTopicReqs)) {
+ return tubeResult;
+ }
+ addTopicReqs.forEach(addTopicReq -> {
+ try {
+ String brokerStr = StringUtils.join(brokerIds, ",");
Review comment:
when finished, we need to test it in a small cluster to make sure it works as expected
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org