You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2020/11/21 11:24:34 UTC

[GitHub] [rocketmq] comdd opened a new issue #2444: AllocateMachineRoomNearby算法存在让consumer负载不均衡的风险

comdd opened a new issue #2444:
URL: https://github.com/apache/rocketmq/issues/2444

The issue tracker is **ONLY** used for bug report(feature request need to follow [RIP process](https://github.com/apache/rocketmq/wiki/RocketMQ-Improvement-Proposal)). Keep in mind, please check whether there is an existing same report before your raise a new one.

Alternately (especially if your communication is not a bug report), you can send mail to our [mailing lists](http://rocketmq.apache.org/about/contact/). We welcome any friendly suggestions, bug fixes, collaboration and other improvements.

Please ensure that your bug report is clear and that it is complete. Otherwise, we may be unable to understand it or to reproduce it, either of which would prevent us from fixing the bug. We strongly recommend the report(bug report or feature request) could include some hints as the following:

**BUG REPORT**

1. Please describe the issue you observed:

- What did you do (The steps to reproduce)?

- What did you expect to see?

- What did you see instead?

2. Please tell us about your environment:

3. Other information (e.g. detailed explanation, logs, related issues, suggestions how to fix, etc):

**FEATURE REQUEST**

1. Please describe the feature you are requesting.
在对messagequeue对应机房上没有consumer的messagequeue做分配时会造成consumer的负载不均衡，比如A、B机房各有3个messagequeue，并且不存在consumer，但是机房C上有4个consumer，在使用该分配策略时，会造成id靠后的consumer永远分配不到这6个messageQueue中的任何一个。
2. Provide any additional detail on your proposed use case for this feature.
可以将所有对应机房上没有consumer的messagequeue整合之后再去面向所有的consumer进行分配，比如上面的case就会变成6个messageQueue分配给4个consumer，可以缓解consumer负载不均衡的情况。
2. Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?

4. If there are some sub-tasks using -[] for each subtask and create a corresponding issue to map to the sub task:

- [sub-task1-issue-number](example_sub_issue1_link_here): sub-task1 description here,
- [sub-task2-issue-number](example_sub_issue2_link_here): sub-task2 description here,
- ...

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] Jaskey commented on issue #2444: AllocateMachineRoomNearby rebalance enhancement request

Posted by GitBox <gi...@apache.org>.

Jaskey commented on issue #2444:
URL: https://github.com/apache/rocketmq/issues/2444#issuecomment-733438276


   I write a test case and reproduce this issue: 
   
   Test : IDCSize = 2 queueSize = 3 consumerSize = 4
   cid: IDC1-CID-0--> res :[MessageQueue [topic=topic_test, brokerName=IDC1-brokerName, queueId=0], MessageQueue [topic=topic_test, brokerName=IDC2-brokerName, queueId=0]]
   cid: IDC1-CID-1--> res :[MessageQueue [topic=topic_test, brokerName=IDC1-brokerName, queueId=1], MessageQueue [topic=topic_test, brokerName=IDC2-brokerName, queueId=1]]
   cid: IDC1-CID-2--> res :[MessageQueue [topic=topic_test, brokerName=IDC1-brokerName, queueId=2], MessageQueue [topic=topic_test, brokerName=IDC2-brokerName, queueId=2]]
   cid: IDC1-CID-3--> res :[]


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] Jaskey removed a comment on issue #2444: AllocateMachineRoomNearby rebalance enhancement request

Posted by GitBox <gi...@apache.org>.

Jaskey removed a comment on issue #2444:
URL: https://github.com/apache/rocketmq/issues/2444#issuecomment-733435242


   This strategy is written by me years ago, would you please provide a test case ?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] Jaskey edited a comment on issue #2444: AllocateMachineRoomNearby rebalance enhancement request

Posted by GitBox <gi...@apache.org>.

Jaskey edited a comment on issue #2444:
URL: https://github.com/apache/rocketmq/issues/2444#issuecomment-733439826


   The reason is that , this strategy is to group the message queues first by machine room, and then allocate them separately, after which it merge the results. So for the IDC1 and IDC2, consumer only see 3 queues in two groups. and 4 consumers are more than the queue numbers, in this case 3, so the result by now is by design. if you have any better implementations, please provide a pr.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] Jaskey commented on issue #2444: AllocateMachineRoomNearby rebalance enhancement request

Posted by GitBox <gi...@apache.org>.

Jaskey commented on issue #2444:
URL: https://github.com/apache/rocketmq/issues/2444#issuecomment-733435242


   This strategy is written by me years ago, would you please provide a test case ?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] Jaskey commented on issue #2444: AllocateMachineRoomNearby rebalance enhancement request

Posted by GitBox <gi...@apache.org>.

Jaskey commented on issue #2444:
URL: https://github.com/apache/rocketmq/issues/2444#issuecomment-733439826


   The reason is that , this strategy is to group the message queue first by machine room, and then allocate them separately, after which it merge the results. So for the IDC1 and IDC2, consumer only see 3 queues in two groups. and 4 consumers are more than the queue numbers, in this case 3, so the result by now is by design. if you have any better implementations, please provide a pr.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org