(druid) branch master updated: Documentation in Basic Cluster Tuning using Group By Metrics (#19083)

[gw...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gwphua pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git


The following commit(s) were added to refs/heads/master by this push:
     new b5e24cc09e0 Documentation in Basic Cluster Tuning using Group By Metrics (#19083)
b5e24cc09e0 is described below

commit b5e24cc09e020c3a6b15949691db24ccc9dc5e11
Author: Virushade <ph...@gmail.com>
AuthorDate: Sun Mar 8 01:25:02 2026 +0800

    Documentation in Basic Cluster Tuning using Group By Metrics (#19083)
    
    * Docs for tuning groupby
    
    * Docs
    
    * Update docs/operations/basic-cluster-tuning.md
    
    Co-authored-by: aho135 <an...@gmail.com>
    
    * SpilledQueries docs
    
    * Update basic-cluster-tuning.md
    
    Co-authored-by: aho135 <an...@gmail.com>
    
    ---------
    
    Co-authored-by: aho135 <an...@gmail.com>
---
 docs/operations/basic-cluster-tuning.md | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/docs/operations/basic-cluster-tuning.md b/docs/operations/basic-cluster-tuning.md
index 4cbd9c964a7..51f8cb0f037 100644
--- a/docs/operations/basic-cluster-tuning.md
+++ b/docs/operations/basic-cluster-tuning.md
@@ -334,6 +334,25 @@ Non-nested GroupBy queries require 1 merge buffer per query, while a nested Grou
 
 The number of merge buffers determines the number of GroupBy queries that can be processed concurrently.
 
+#### Using metrics to tune GroupBy buffer configuration
+
+Druid can emit metrics that help you right-size merge buffers and related GroupBy configuration. These metrics are available when the `GroupByStatsMonitor` module is enabled by adding `org.apache.druid.server.metrics.GroupByStatsMonitor` to `druid.monitoring.monitors`. See the [metrics reference](metrics.md) for full details.
+
+##### Sizing `druid.processing.buffer.sizeBytes`
+
+- `mergeBuffer/maxBytesUsed`: peak merge buffer bytes used by any single GroupBy query within the emission period. If this value consistently approaches `druid.processing.buffer.sizeBytes`, consider increasing the buffer size.
+- `groupBy/maxSpilledBytes`: peak bytes spilled to disk by any single GroupBy query. Non-zero values indicate that merge buffers are too small to hold intermediate results in memory, causing disk spill. Increasing `druid.processing.buffer.sizeBytes` reduces spilling. You can also adjust `druid.query.groupBy.maxOnDiskStorage` to control how much spilling is allowed before a query fails.
+- `groupBy/spilledQueries`: number of GroupBy queries spilled to disk within the emission period. Non-zero values may indicate that your buffer size is too small and should be increased to avoid performance issues caused by excessive spilling.
+
+##### Sizing `druid.processing.numMergeBuffers`
+
+- `mergeBuffer/pendingRequests`: number of queries waiting to acquire merge buffers. Persistently non-zero values indicate merge buffer pool exhaustion; consider increasing `druid.processing.numMergeBuffers`.
+- `mergeBuffer/maxAcquisitionTimeNs`: peak time in nanoseconds that any single GroupBy query waited to acquire merge buffers. High values suggest contention on the merge buffer pool; increasing `druid.processing.numMergeBuffers` can reduce wait times.
+
+##### Sizing `druid.query.groupBy.maxMergingDictionarySize`
+
+- `groupBy/maxMergeDictionarySize`: peak on-heap merge dictionary size in bytes for any single GroupBy query. If this approaches `druid.query.groupBy.maxMergingDictionarySize`, queries may spill to disk if `druid.query.groupBy.maxOnDiskStorage` is configured or fail. If this happens, consider increasing the dictionary size limit.
+
 <a name="connection-pool"></a>
 
 ## Connection pool guidelines


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org