[GitHub] leventov opened a new issue #7143: Delegate cost computation in CostBalancerStrategy to the common ForkJoinPool and remove balancerComputeThreads config

[GitBox <gi...@apache.org>]

leventov opened a new issue #7143: Delegate cost computation in CostBalancerStrategy to the common ForkJoinPool and remove balancerComputeThreads config
URL: https://github.com/apache/incubator-druid/issues/7143
 
 
   I don't see a point of using custom thread pool for `CostBalancerStrategy`, because it delegates to the pool "pure", non-blocking `cost()` calls. When methods are non-blocking, it's a job for the common `ForkJoinPool` via parallel Streams.
   
   This will likely also improve scalability because instead of wrapping each `cost()` call as a separate task for a thread pool (this might be the reason for poor scalability - see #3275), parallel `Stream` won't execute more slices than there are cores in the system.
   
   In the transition period, the `balancerComputeThreads` may be ignored and a warning or an alert is issued if this option is found in the config.
   
   FYI @himadrisingh001 @xvrl @fjy - contributors to the original change that introduced this thread pool (#399).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org