You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/12/07 17:32:41 UTC
[GitHub] [druid] jihoonson commented on a change in pull request #11960: clean up the balancing code around the batched vs deprecated way of sampling segments to balance
jihoonson commented on a change in pull request #11960:
URL: https://github.com/apache/druid/pull/11960#discussion_r764219324
##########
File path: web-console/src/druid-models/coordinator-dynamic-config.tsx
##########
@@ -213,14 +213,16 @@ export const COORDINATOR_DYNAMIC_CONFIG_FIELDS: Field<CoordinatorDynamicConfig>[
defaultValue: 100,
info: (
<>
- The percentage of the total number of segments in the cluster that are considered every time
- a segment needs to be selected for a move. Druid orders servers by available capacity
- ascending (the least available capacity first) and then iterates over the servers. For each
- server, Druid iterates over the segments on the server, considering them for moving. The
- default config of 100% means that every segment on every server is a candidate to be moved.
- This should make sense for most small to medium-sized clusters. However, an admin may find
- it preferable to drop this value lower if they don't think that it is worthwhile to
- consider every single segment in the cluster each time it is looking for a segment to move.
+ Deprecated. This will be phased out by the batched segment sampler. Only used if
Review comment:
Can we document what the preferred alternative is here as well?
##########
File path: docs/configuration/index.md
##########
@@ -886,7 +886,7 @@ Issuing a GET request at the same URL will return the spec that is currently in
|`mergeSegmentsLimit`|The maximum number of segments that can be in a single [append task](../ingestion/tasks.md).|100|
|`maxSegmentsToMove`|The maximum number of segments that can be moved at any given time.|5|
|`useBatchedSegmentSampler`|Boolean flag for whether or not we should use the Reservoir Sampling with a reservoir of size k instead of fixed size 1 to pick segments to move. This option can be enabled to speed up segment balancing process, especially if there are huge number of segments in the cluster or if there are too many segments to move.|false|
-|`percentOfSegmentsToConsiderPerMove`|The percentage of the total number of segments in the cluster that are considered every time a segment needs to be selected for a move. Druid orders servers by available capacity ascending (the least available capacity first) and then iterates over the servers. For each server, Druid iterates over the segments on the server, considering them for moving. The default config of 100% means that every segment on every server is a candidate to be moved. This should make sense for most small to medium-sized clusters. However, an admin may find it preferable to drop this value lower if they don't think that it is worthwhile to consider every single segment in the cluster each time it is looking for a segment to move.|100|
+|`percentOfSegmentsToConsiderPerMove`|Deprecated. Will eventually be phased out by the batched segment sampler. Only used if `useBatchedSegmentSampler == false`. The percentage of the total number of segments in the cluster that are considered every time a segment needs to be selected for a move. Druid orders servers by available capacity ascending (the least available capacity first) and then iterates over the servers. For each server, Druid iterates over the segments on the server, considering them for moving. The default config of 100% means that every segment on every server is a candidate to be moved. This should make sense for most small to medium-sized clusters. However, an admin may find it preferable to drop this value lower if they don't think that it is worthwhile to consider every single segment in the cluster each time it is looking for a segment to move.|100|
Review comment:
Can we document what the preferred alternative is here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org