You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@celeborn.apache.org by "gaoyajun02 (via GitHub)" <gi...@apache.org> on 2025/01/20 12:31:02 UTC
[PR] [CELEBORN-1843] Optimize roundrobin for more balanced disk slot allocation [celeborn]
gaoyajun02 opened a new pull request, #3074:
URL: https://github.com/apache/celeborn/pull/3074
### What changes were proposed in this pull request?
This PR optimizes the RoundRobin algorithm to achieve a more balanced disk slot allocation across workers.
Previously, when allocating 3000 partitions using RoundRobin, the slot distribution across worker disks was [668, 666, 666], which resulted in one disk having 2 more slots than the others.
After the optimization, the slot distribution is now [667, 667, 666], ensuring a more balanced allocation.
### Why are the changes needed?
The changes are necessary to improve load balancing across worker disks, reducing the risk of overloading a single disk. This ensures a more predictable and fair distribution of slots, which can lead to better performance and resource utilization.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
UT
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CELEBORN-1843] Optimize roundrobin for more balanced disk slot allocation [celeborn]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #3074:
URL: https://github.com/apache/celeborn/pull/3074#issuecomment-2650121616
This PR is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 10 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CELEBORN-1843] Optimize roundrobin for more balanced disk slot allocation [celeborn]
Posted by "RexXiong (via GitHub)" <gi...@apache.org>.
RexXiong closed pull request #3074: [CELEBORN-1843] Optimize roundrobin for more balanced disk slot allocation
URL: https://github.com/apache/celeborn/pull/3074
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CELEBORN-1843] Optimize roundrobin for more balanced disk slot allocation [celeborn]
Posted by "RexXiong (via GitHub)" <gi...@apache.org>.
RexXiong commented on PR #3074:
URL: https://github.com/apache/celeborn/pull/3074#issuecomment-2652783184
Thanks merge to main(v0.6.0)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [CELEBORN-1843] Optimize roundrobin for more balanced disk slot allocation [celeborn]
Posted by "FMX (via GitHub)" <gi...@apache.org>.
FMX commented on PR #3074:
URL: https://github.com/apache/celeborn/pull/3074#issuecomment-2603892624
This PR looks good to me, but it should be subject to performance tests to ensure that nothing unexpected happens. The slot distribution will affect the performance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org