You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/09/17 12:07:08 UTC
(datafusion) branch asf-site updated: Publish built docs triggered by 790c46383084537e90e5b292c50d6b7711b40f51
This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 98bdd2d510 Publish built docs triggered by 790c46383084537e90e5b292c50d6b7711b40f51
98bdd2d510 is described below
commit 98bdd2d5102828ab4bc85acb4b36d1f4174ddd75
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Thu Sep 17 12:07:03 2026 +0000
Publish built docs triggered by 790c46383084537e90e5b292c50d6b7711b40f51
---
_sources/user-guide/metrics.md.txt | 19 +++++++++++++++++++
searchindex.js | 2 +-
user-guide/metrics.html | 18 ++++++++++++++++++
3 files changed, 38 insertions(+), 1 deletion(-)
diff --git a/_sources/user-guide/metrics.md.txt b/_sources/user-guide/metrics.md.txt
index 5bb4895a3d..111df66ccc 100644
--- a/_sources/user-guide/metrics.md.txt
+++ b/_sources/user-guide/metrics.md.txt
@@ -154,6 +154,25 @@ when it is evaluated per aggregate. The legacy grouped hash path evaluates
filters collectively, so its per-aggregate `arguments` timers cover argument
expressions only; their sum need not equal `aggregate_arguments_time`.
+Aggregate implementations can also expose optional internal submetrics. These
+use the `agg_expr_{index}_internal_{subphase}_time` naming and `aggregate`
+label. The `internal` segment keeps them separate from call-boundary timers;
+`subphase` is a stable identifier owned and documented by the aggregate
+implementation. They are registered lazily only when an aggregate requests
+them, so aggregates without internal submetrics add no metrics. Registration
+is per `(aggregate expression index, subphase, partition)`: replacement
+accumulators in that partition share the same time, and normal metric display
+combines that time across partitions. An aggregate may request its submetric
+during accumulator construction, so it can appear even when its input is empty.
+For example, `array_agg(DISTINCT ...)` records the time spent deduplicating
+input values as `agg_expr_{index}_internal_distinct_time`. Grouped accumulation
+records this once per input batch, rather than once per group, to avoid making
+metric collection proportional to group cardinality. These submetrics complement
+the `update`, `merge`, `state`, and `evaluate` timers rather than subdividing or
+replacing them. An internal submetric may therefore overlap its enclosing phase
+timer; it is a supplementary diagnostic and must not be added to phase timings
+as a breakdown.
+
Except for the `Summary` metric `reduction_factor`, these operator-level and
per-aggregate metrics are `Dev` metrics. They appear in `EXPLAIN ANALYZE` when
`datafusion.explain.analyze_level` includes `Dev` (the default), but are omitted
diff --git a/searchindex.js b/searchindex.js
index 5d49b37acd..2d14224c42 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles":{"!=":[[77,"op-neq"]],"!~":[[77,"op-re-not-match"]],"!~*":[[77,"op-re-not-match-i"]],"!~~":[[77,"id19"]],"!~~*":[[77,"id20"]],"#":[[77,"op-bit-xor"]],"%":[[77,"op-modulo"]],"&":[[77,"op-bit-and"]],"(relation, name) tuples in logical fields and logical columns are unique":[[16,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[77,"op-multiply"]],"+":[[77,"op-plus"]],"-":[[77,"op-minus"]],"/":[[77,"op-divide"]],"1. Array Literal Con [...]
\ No newline at end of file
+Search.setIndex({"alltitles":{"!=":[[77,"op-neq"]],"!~":[[77,"op-re-not-match"]],"!~*":[[77,"op-re-not-match-i"]],"!~~":[[77,"id19"]],"!~~*":[[77,"id20"]],"#":[[77,"op-bit-xor"]],"%":[[77,"op-modulo"]],"&":[[77,"op-bit-and"]],"(relation, name) tuples in logical fields and logical columns are unique":[[16,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[77,"op-multiply"]],"+":[[77,"op-plus"]],"-":[[77,"op-minus"]],"/":[[77,"op-divide"]],"1. Array Literal Con [...]
\ No newline at end of file
diff --git a/user-guide/metrics.html b/user-guide/metrics.html
index 9b84f4cc02..c9afba7966 100644
--- a/user-guide/metrics.html
+++ b/user-guide/metrics.html
@@ -674,6 +674,24 @@ rather than using accumulators.</p>
when it is evaluated per aggregate. The legacy grouped hash path evaluates
filters collectively, so its per-aggregate <code class="docutils literal notranslate"><span class="pre">arguments</span></code> timers cover argument
expressions only; their sum need not equal <code class="docutils literal notranslate"><span class="pre">aggregate_arguments_time</span></code>.</p>
+<p>Aggregate implementations can also expose optional internal submetrics. These
+use the <code class="docutils literal notranslate"><span class="pre">agg_expr_{index}_internal_{subphase}_time</span></code> naming and <code class="docutils literal notranslate"><span class="pre">aggregate</span></code>
+label. The <code class="docutils literal notranslate"><span class="pre">internal</span></code> segment keeps them separate from call-boundary timers;
+<code class="docutils literal notranslate"><span class="pre">subphase</span></code> is a stable identifier owned and documented by the aggregate
+implementation. They are registered lazily only when an aggregate requests
+them, so aggregates without internal submetrics add no metrics. Registration
+is per <code class="docutils literal notranslate"><span class="pre">(aggregate</span> <span class="pre">expression</span> <span class="pre">index,</span> <span class="pre">subphase,</span> <span class="pre">partition)</span></code>: replacement
+accumulators in that partition share the same time, and normal metric display
+combines that time across partitions. An aggregate may request its submetric
+during accumulator construction, so it can appear even when its input is empty.
+For example, <code class="docutils literal notranslate"><span class="pre">array_agg(DISTINCT</span> <span class="pre">...)</span></code> records the time spent deduplicating
+input values as <code class="docutils literal notranslate"><span class="pre">agg_expr_{index}_internal_distinct_time</span></code>. Grouped accumulation
+records this once per input batch, rather than once per group, to avoid making
+metric collection proportional to group cardinality. These submetrics complement
+the <code class="docutils literal notranslate"><span class="pre">update</span></code>, <code class="docutils literal notranslate"><span class="pre">merge</span></code>, <code class="docutils literal notranslate"><span class="pre">state</span></code>, and <code class="docutils literal notranslate"><span class="pre">evaluate</span></code> timers rather than subdividing or
+replacing them. An internal submetric may therefore overlap its enclosing phase
+timer; it is a supplementary diagnostic and must not be added to phase timings
+as a breakdown.</p>
<p>Except for the <code class="docutils literal notranslate"><span class="pre">Summary</span></code> metric <code class="docutils literal notranslate"><span class="pre">reduction_factor</span></code>, these operator-level and
per-aggregate metrics are <code class="docutils literal notranslate"><span class="pre">Dev</span></code> metrics. They appear in <code class="docutils literal notranslate"><span class="pre">EXPLAIN</span> <span class="pre">ANALYZE</span></code> when
<code class="docutils literal notranslate"><span class="pre">datafusion.explain.analyze_level</span></code> includes <code class="docutils literal notranslate"><span class="pre">Dev</span></code> (the default), but are omitted
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org