You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "Adithya-Shetty100 (via GitHub)" <gi...@apache.org> on 2026/06/17 07:46:16 UTC
[PR] docs: document druid.broker.realtime.balancer.type and select.tier (druid)
Adithya-Shetty100 opened a new pull request, #19591:
URL: https://github.com/apache/druid/pull/19591
### Description
[#19062](https://github.com/apache/druid/pull/19062) added two new Broker configuration properties that let operators override the Broker's balancer and tier selection strategies for realtime servers:
- `druid.broker.realtime.balancer.type`
- `druid.broker.realtime.select.tier`
These properties were merged without documentation (the author noted they could be documented separately). This PR documents them in the Broker query configuration table.
#### What changed
Added two rows to the Broker query configuration table:
- `druid.broker.realtime.balancer.type` — possible values `random`, `connectionCount`; overrides `druid.broker.balancer.type` for realtime servers only; falls back to `druid.broker.balancer.type` when unset.
- `druid.broker.realtime.select.tier` — possible values `highestPriority`, `lowestPriority`, `custom`, `preferred`; overrides `druid.broker.select.tier` for realtime servers only; falls back to `druid.broker.select.tier` when unset. The description also notes that the corresponding `druid.broker.realtime.select.tier.*` sub-configs behave the same as their `druid.broker.select.tier.*` counterparts.
The behavior documented here is taken directly from `BrokerRealtimeSelectorModule`.
#### Release note
Documented the `druid.broker.realtime.balancer.type` and `druid.broker.realtime.select.tier` Broker properties introduced in #19062.
<hr>
This PR has:
- [x] been self-reviewed.
- [x] added documentation for new or modified features or behaviors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] docs: document druid.broker.realtime.balancer.type and select.tier (druid)
Posted by "FrankChen021 (via GitHub)" <gi...@apache.org>.
FrankChen021 commented on code in PR #19591:
URL: https://github.com/apache/druid/pull/19591#discussion_r3427977326
##########
docs/configuration/index.md:
##########
@@ -1645,6 +1645,8 @@ These Broker configurations can be defined in the `broker/runtime.properties` fi
|`druid.broker.select.tier.custom.priorities`|An array of integer priorities, such as `[-1, 0, 1, 2]`|Select servers in tiers with a custom priority list.|The config only has effect if `druid.broker.select.tier` is set to `custom`. If `druid.broker.select.tier` is set to `custom` but this config is not specified, the effect is the same as `druid.broker.select.tier` set to `highestPriority`. Any of the integers in this config can be ignored if there's no corresponding tiers with such priorities. Tiers with priorities explicitly specified in this config always have higher priority than those not and those not specified fall back to use `highestPriority` strategy among themselves.|
|`druid.broker.select.tier.preferred.tier`| The preferred tier name. E.g., `_default_tier` | A non-empty value that specifies the preferred tier in which historical servers will be picked up for queries. If there are not enough historical servers from the preferred tier, servers from other tiers (if there are any) will be selected. This config only has effect if `druid.broker.select.tier` is set to `preferred` | null |
|`druid.broker.select.tier.preferred.priority`| `highest`, `lowest` | If there are multiple candidates in a preferred tier, specifies the priority to pick up candidates. By default, the higher priority a historical, the higher chances it will be picked up. This config only has effect if `druid.broker.select.tier` is set to `preferred`| `highest` |
+|`druid.broker.realtime.balancer.type`|`random`, `connectionCount`|Determines how the broker balances connections to realtime processes. This overrides `druid.broker.balancer.type` for realtime servers only; Historical processes are unaffected. If not set, realtime servers use the strategy configured by `druid.broker.balancer.type`.|Value of `druid.broker.balancer.type`|
Review Comment:
[P2] Realtime balancer override is documented as standalone
This row says `druid.broker.realtime.balancer.type` overrides the historical balancer for realtime servers, but `RealtimeTierSelectorStrategyProvider` only constructs a realtime tier selector with the realtime `ServerSelectorStrategy` when `druid.broker.realtime.select.tier` is set. If only the balancer override is configured, it returns the default historical `TierSelectorStrategy`, so realtime server picking still uses the default balancer. Either fix that implementation path or document the dependency so operators do not configure a no-op override.
##########
docs/configuration/index.md:
##########
@@ -1645,6 +1645,8 @@ These Broker configurations can be defined in the `broker/runtime.properties` fi
|`druid.broker.select.tier.custom.priorities`|An array of integer priorities, such as `[-1, 0, 1, 2]`|Select servers in tiers with a custom priority list.|The config only has effect if `druid.broker.select.tier` is set to `custom`. If `druid.broker.select.tier` is set to `custom` but this config is not specified, the effect is the same as `druid.broker.select.tier` set to `highestPriority`. Any of the integers in this config can be ignored if there's no corresponding tiers with such priorities. Tiers with priorities explicitly specified in this config always have higher priority than those not and those not specified fall back to use `highestPriority` strategy among themselves.|
|`druid.broker.select.tier.preferred.tier`| The preferred tier name. E.g., `_default_tier` | A non-empty value that specifies the preferred tier in which historical servers will be picked up for queries. If there are not enough historical servers from the preferred tier, servers from other tiers (if there are any) will be selected. This config only has effect if `druid.broker.select.tier` is set to `preferred` | null |
|`druid.broker.select.tier.preferred.priority`| `highest`, `lowest` | If there are multiple candidates in a preferred tier, specifies the priority to pick up candidates. By default, the higher priority a historical, the higher chances it will be picked up. This config only has effect if `druid.broker.select.tier` is set to `preferred`| `highest` |
+|`druid.broker.realtime.balancer.type`|`random`, `connectionCount`|Determines how the broker balances connections to realtime processes. This overrides `druid.broker.balancer.type` for realtime servers only; Historical processes are unaffected. If not set, realtime servers use the strategy configured by `druid.broker.balancer.type`.|Value of `druid.broker.balancer.type`|
+|`druid.broker.realtime.select.tier`|`highestPriority`, `lowestPriority`, `custom`, `preferred`|Tier selection strategy for realtime processes. This overrides `druid.broker.select.tier` for realtime servers only; Historical processes are unaffected. If not set, realtime servers use the strategy configured by `druid.broker.select.tier`. The corresponding sub-configs `druid.broker.realtime.select.tier.custom.priorities`, `druid.broker.realtime.select.tier.preferred.tier`, and `druid.broker.realtime.select.tier.preferred.priority` behave the same as their `druid.broker.select.tier.*` counterparts.|Value of `druid.broker.select.tier`|
Review Comment:
[P3] Supported realtime tier strategies are omitted
`BrokerRealtimeSelectorModule` and `TierSelectorStrategy` also support `strict` and `pooled` for `druid.broker.realtime.select.tier`, and the tests exercise `druid.broker.realtime.select.tier.strict.priorities` and `druid.broker.realtime.select.tier.pooled.priorities`. The new docs row lists only `highestPriority`, `lowestPriority`, `custom`, and `preferred`, and only mentions the custom/preferred sub-configs, so valid realtime selector configurations remain undocumented.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] docs: document druid.broker.realtime.balancer.type and select.tier (druid)
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19591:
URL: https://github.com/apache/druid/pull/19591#issuecomment-5403217491
This pull request has been marked as stale due to 60 days of inactivity.
It will be closed in 4 weeks if no further activity occurs. If you think
that's incorrect or this pull request should instead be reviewed, please simply
write any comment. Even if closed, you can still revive the PR at any time or
discuss it on the dev@druid.apache.org list.
Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] docs: document druid.broker.realtime.balancer.type and select.tier (druid)
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #19591:
URL: https://github.com/apache/druid/pull/19591#issuecomment-5769582288
This pull request/issue has been closed due to lack of activity. If you think that
is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] docs: document druid.broker.realtime.balancer.type and select.tier (druid)
Posted by "abhishekrb19 (via GitHub)" <gi...@apache.org>.
abhishekrb19 commented on code in PR #19591:
URL: https://github.com/apache/druid/pull/19591#discussion_r3471539770
##########
docs/configuration/index.md:
##########
@@ -1645,6 +1645,8 @@ These Broker configurations can be defined in the `broker/runtime.properties` fi
|`druid.broker.select.tier.custom.priorities`|An array of integer priorities, such as `[-1, 0, 1, 2]`|Select servers in tiers with a custom priority list.|The config only has effect if `druid.broker.select.tier` is set to `custom`. If `druid.broker.select.tier` is set to `custom` but this config is not specified, the effect is the same as `druid.broker.select.tier` set to `highestPriority`. Any of the integers in this config can be ignored if there's no corresponding tiers with such priorities. Tiers with priorities explicitly specified in this config always have higher priority than those not and those not specified fall back to use `highestPriority` strategy among themselves.|
|`druid.broker.select.tier.preferred.tier`| The preferred tier name. E.g., `_default_tier` | A non-empty value that specifies the preferred tier in which historical servers will be picked up for queries. If there are not enough historical servers from the preferred tier, servers from other tiers (if there are any) will be selected. This config only has effect if `druid.broker.select.tier` is set to `preferred` | null |
|`druid.broker.select.tier.preferred.priority`| `highest`, `lowest` | If there are multiple candidates in a preferred tier, specifies the priority to pick up candidates. By default, the higher priority a historical, the higher chances it will be picked up. This config only has effect if `druid.broker.select.tier` is set to `preferred`| `highest` |
+|`druid.broker.realtime.balancer.type`|`random`, `connectionCount`|Determines how the broker balances connections to realtime processes. This overrides `druid.broker.balancer.type` for realtime servers only; Historical processes are unaffected. If not set, realtime servers use the strategy configured by `druid.broker.balancer.type`.|Value of `druid.broker.balancer.type`|
+|`druid.broker.realtime.select.tier`|`highestPriority`, `lowestPriority`, `custom`, `preferred`|Tier selection strategy for realtime processes. This overrides `druid.broker.select.tier` for realtime servers only; Historical processes are unaffected. If not set, realtime servers use the strategy configured by `druid.broker.select.tier`. The corresponding sub-configs `druid.broker.realtime.select.tier.custom.priorities`, `druid.broker.realtime.select.tier.preferred.tier`, and `druid.broker.realtime.select.tier.preferred.priority` behave the same as their `druid.broker.select.tier.*` counterparts.|Value of `druid.broker.select.tier`|
Review Comment:
Yeah, this is on me for not documenting the new strategies added in https://github.com/apache/druid/pull/19094.
@Adithya-Shetty100, do you want to include them here for completeness? Otherwise, I can handle that separately as well. We’ve been running these strategies in production for sometime now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] docs: document druid.broker.realtime.balancer.type and select.tier (druid)
Posted by "abhishekrb19 (via GitHub)" <gi...@apache.org>.
abhishekrb19 commented on code in PR #19591:
URL: https://github.com/apache/druid/pull/19591#discussion_r3471589011
##########
docs/configuration/index.md:
##########
@@ -1645,6 +1645,8 @@ These Broker configurations can be defined in the `broker/runtime.properties` fi
|`druid.broker.select.tier.custom.priorities`|An array of integer priorities, such as `[-1, 0, 1, 2]`|Select servers in tiers with a custom priority list.|The config only has effect if `druid.broker.select.tier` is set to `custom`. If `druid.broker.select.tier` is set to `custom` but this config is not specified, the effect is the same as `druid.broker.select.tier` set to `highestPriority`. Any of the integers in this config can be ignored if there's no corresponding tiers with such priorities. Tiers with priorities explicitly specified in this config always have higher priority than those not and those not specified fall back to use `highestPriority` strategy among themselves.|
|`druid.broker.select.tier.preferred.tier`| The preferred tier name. E.g., `_default_tier` | A non-empty value that specifies the preferred tier in which historical servers will be picked up for queries. If there are not enough historical servers from the preferred tier, servers from other tiers (if there are any) will be selected. This config only has effect if `druid.broker.select.tier` is set to `preferred` | null |
|`druid.broker.select.tier.preferred.priority`| `highest`, `lowest` | If there are multiple candidates in a preferred tier, specifies the priority to pick up candidates. By default, the higher priority a historical, the higher chances it will be picked up. This config only has effect if `druid.broker.select.tier` is set to `preferred`| `highest` |
+|`druid.broker.realtime.balancer.type`|`random`, `connectionCount`|Determines how the broker balances connections to realtime processes. This overrides `druid.broker.balancer.type` for realtime servers only; Historical processes are unaffected. If not set, realtime servers use the strategy configured by `druid.broker.balancer.type`.|Value of `druid.broker.balancer.type`|
Review Comment:
This does seem like a bug when `druid.broker.realtime.balancer.type` is overridden without `druid.broker.realtime.select.tier` is set
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] docs: document druid.broker.realtime.balancer.type and select.tier (druid)
Posted by "abhishekrb19 (via GitHub)" <gi...@apache.org>.
abhishekrb19 commented on code in PR #19591:
URL: https://github.com/apache/druid/pull/19591#discussion_r3471539770
##########
docs/configuration/index.md:
##########
@@ -1645,6 +1645,8 @@ These Broker configurations can be defined in the `broker/runtime.properties` fi
|`druid.broker.select.tier.custom.priorities`|An array of integer priorities, such as `[-1, 0, 1, 2]`|Select servers in tiers with a custom priority list.|The config only has effect if `druid.broker.select.tier` is set to `custom`. If `druid.broker.select.tier` is set to `custom` but this config is not specified, the effect is the same as `druid.broker.select.tier` set to `highestPriority`. Any of the integers in this config can be ignored if there's no corresponding tiers with such priorities. Tiers with priorities explicitly specified in this config always have higher priority than those not and those not specified fall back to use `highestPriority` strategy among themselves.|
|`druid.broker.select.tier.preferred.tier`| The preferred tier name. E.g., `_default_tier` | A non-empty value that specifies the preferred tier in which historical servers will be picked up for queries. If there are not enough historical servers from the preferred tier, servers from other tiers (if there are any) will be selected. This config only has effect if `druid.broker.select.tier` is set to `preferred` | null |
|`druid.broker.select.tier.preferred.priority`| `highest`, `lowest` | If there are multiple candidates in a preferred tier, specifies the priority to pick up candidates. By default, the higher priority a historical, the higher chances it will be picked up. This config only has effect if `druid.broker.select.tier` is set to `preferred`| `highest` |
+|`druid.broker.realtime.balancer.type`|`random`, `connectionCount`|Determines how the broker balances connections to realtime processes. This overrides `druid.broker.balancer.type` for realtime servers only; Historical processes are unaffected. If not set, realtime servers use the strategy configured by `druid.broker.balancer.type`.|Value of `druid.broker.balancer.type`|
+|`druid.broker.realtime.select.tier`|`highestPriority`, `lowestPriority`, `custom`, `preferred`|Tier selection strategy for realtime processes. This overrides `druid.broker.select.tier` for realtime servers only; Historical processes are unaffected. If not set, realtime servers use the strategy configured by `druid.broker.select.tier`. The corresponding sub-configs `druid.broker.realtime.select.tier.custom.priorities`, `druid.broker.realtime.select.tier.preferred.tier`, and `druid.broker.realtime.select.tier.preferred.priority` behave the same as their `druid.broker.select.tier.*` counterparts.|Value of `druid.broker.select.tier`|
Review Comment:
Yeah, this is on me for not documenting the new strategies added in https://github.com/apache/druid/pull/19094.
These strategies apply to and are configurable on both the historicals and realtime servers with replicas.
@Adithya-Shetty100, do you want to include them here for completeness? Otherwise, I can handle that separately as well. We’ve been running these strategies in production for sometime now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] docs: document druid.broker.realtime.balancer.type and select.tier (druid)
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #19591: docs: document druid.broker.realtime.balancer.type and select.tier
URL: https://github.com/apache/druid/pull/19591
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org