You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by GitBox <gi...@apache.org> on 2021/02/22 18:26:03 UTC
[GitHub] [helix] jiajunwang opened a new issue #1653: Gracefully handle very large partitions in WAGED
jiajunwang opened a new issue #1653:
URL: https://github.com/apache/helix/issues/1653
**Is your feature request related to a problem? Please describe.**
It has been detected that a very large partition may impact WAGED rebalance performance. For example, a partition that is weighted more than 50% of the instance capacity might not be able to re-allocated without lifting the other smaller partitions. For now, WAGED won't do this type of cascading re-allocating. As a result, the rebalance calculation will fail and the large partition won't be re-allocated even it is necessary.
**Describe the solution you'd like**
My proposal to handle this situation is that,
1. WAGED should read and respect resource priority. If it is configured, WAGED should always rebalance the higher priority resource partitions first.
2. If there is no enough room for the high priority resource partition to be re-allocation (but overall cluster capacity allows), then the rebalancer shall lift lower priority resources to accommodate the high prioritized one.
**Additional context**
Add any other context or screenshots about the feature request here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] jiajunwang commented on issue #1653: Gracefully handle very large partitions in WAGED
Posted by GitBox <gi...@apache.org>.
jiajunwang commented on issue #1653:
URL: https://github.com/apache/helix/issues/1653#issuecomment-789377371
In another related but different case, if the partitions in one resource have very different weight, then even the resource priority does not help.
This indicates that the 2nd logic we mentioned above should also be applied to a wider range. Not only for the lower priority resources.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org