You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uniffle.apache.org by "advancedxy (via GitHub)" <gi...@apache.org> on 2023/02/22 06:17:28 UTC
[GitHub] [incubator-uniffle] advancedxy opened a new issue, #642: [Improvement] better default shuffle server configuration
advancedxy opened a new issue, #642:
URL: https://github.com/apache/incubator-uniffle/issues/642
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.
### What would you like to be improved?
1. `rss.server.buffer.capacity` could have a default settings, `JVM MaxHeapSize` * some factor
2. `rss.server.read.buffer.capacity` is similar with `rss.server.buffer.capacity`
Otherwise, once the max heap size of shuffle server is changed, users/admins have to reconfigure these two buffer settings.
----
1. `rss.server.disk.capacity` is -1(by default), shuffle server uses the total space of one disk, which seems operation unfriendly. Shuffle server should reserve some partition of disk space and not use all the disks, such as at most 95% the capacity.
### How should we improve?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] zuston commented on issue #642: [Improvement] better default shuffle server configuration
Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston commented on issue #642:
URL: https://github.com/apache/incubator-uniffle/issues/642#issuecomment-1444969108
> rss.server.disk.capacity is -1(by default), shuffle server uses the total space of one disk, which seems operation unfriendly. Shuffle server should reserve some partition of disk space and not use all the disks, such as at most 95% the capacity.
I think this config as a ratio value will be better, the current concrete disk capacity is not flexible especially for heterogeneous storage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] advancedxy commented on issue #642: [Improvement] better default shuffle server configuration
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on issue #642:
URL: https://github.com/apache/incubator-uniffle/issues/642#issuecomment-1439496028
cc @jerqi, WDYT? would you like to refine this settings?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] advancedxy closed issue #642: [Improvement] better default shuffle server configuration
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy closed issue #642: [Improvement] better default shuffle server configuration
URL: https://github.com/apache/incubator-uniffle/issues/642
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] jerqi commented on issue #642: [Improvement] better default shuffle server configuration
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on issue #642:
URL: https://github.com/apache/incubator-uniffle/issues/642#issuecomment-1439508032
> cc @jerqi, WDYT? would you like to refine this settings?
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] advancedxy commented on issue #642: [Improvement] better default shuffle server configuration
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on issue #642:
URL: https://github.com/apache/incubator-uniffle/issues/642#issuecomment-1444993482
> > rss.server.disk.capacity is -1(by default), shuffle server uses the total space of one disk, which seems operation unfriendly. Shuffle server should reserve some partition of disk space and not use all the disks, such as at most 95% the capacity.
>
> I think this config as a ratio value will be better, the current concrete disk capacity is not flexible especially for heterogeneous storage. @advancedxy
:+1. Sound good to me. But the fixed capacity has its own use cases, I will not remove this configuration, rather add a ratio configuration then use it preferably.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org