You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by yu...@apache.org on 2026/10/09 11:50:48 UTC
(doris) branch master updated: [fix](ivm) Materialize aggregate state columns dropped by expressions over aggregates (#68787)
This is an automated email from the ASF dual-hosted git repository.
yujun777 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new cca5b92a3ec [fix](ivm) Materialize aggregate state columns dropped by expressions over aggregates (#68787)
cca5b92a3ec is described below
commit cca5b92a3ecbbd191fd803bc57fe98bfce15b1e7
Author: yujun <yu...@selectdb.com>
AuthorDate: Fri Oct 9 19:50:39 2026 +0800
[fix](ivm) Materialize aggregate state columns dropped by expressions over aggregates (#68787)
An IVM materialized view whose select list wraps an aggregate in a
scalar expression, for example
```sql
CREATE MATERIALIZED VIEW mv REFRESH INCREMENTAL AS SELECT k, SUM(v) * 100 AS s100 FROM t GROUP BY k;
```
is accepted and completes its first refresh, but every later incremental
refresh fails with
`reason=PLAN_REWRITE_FAILED, detail=IVM failed to find slot: sum(v)`.
Apply merges the old MV state in the state domain and then re-applies
the outer expression:
`new.s100 = f(apply(old_mv.sum_v, delta.sum_v))`. That requires the MV
to persist a column carrying
`SUM(v)` itself, because apply reads the old state by column name from
the MV physical table. `SUM`,
`COUNT(expr)`, `MIN`/`MAX`, `COLLECT_LIST`/`ARRAY_AGG` and
`BITMAP_UNION` keep that mergeable state
in their *visible* column, and normalize drops that column as soon as an
upper expression consumes it
without projecting it. `AVG`, `BITMAP_UNION_COUNT` and `COUNT(*)` are
unaffected because their state
is already a hidden state column, which is why only some aggregates
broke. A view of this shape is
incremental-capable, so adding the missing state column is the fix;
rejecting the shape at CREATE
would cut a capability the engine already has.
Fix: normalize materializes the dropped state column as a hidden
pass-through carrier at the project
layer that drops it, and rebinds every target that reads it, including
hidden state the column pool
had shared with a dropped visible column (AVG's hidden SUM). The carrier
is named exactly as the
delta sub-plan already names that state, so delta aggregate outputs,
delta slot lookups and the write
path are untouched, and shapes that already work gain no column:
- `SELECT k, SUM(v) * 100` gains `__DORIS_IVM_AGG_0_SUM_COL__` (plus the
existing hidden non-NULL count)
- `SELECT k, SUM(v) * 100, AVG(v) * 200` gains one carrier shared by SUM
and AVG's hidden SUM
- `SELECT k, SUM(v) / COUNT(v)` gains one SUM carrier and one COUNT
carrier for both targets
- `SELECT k, SUM(v) AS s, SUM(v) * 100`, `SUM(v * 100)`, `COUNT(*) * 2`,
`ROUND(AVG(v), 2)` and
`BITMAP_UNION_COUNT(TO_BITMAP(v)) + 0` keep their exact previous layout
Normalize additionally validates that every state column apply reads
survived into the MV output, so
a missing state column fails at CREATE/refresh instead of at the first
incremental refresh.
---
.../main/java/org/apache/doris/mtmv/ivm/AGENTS.md | 2 +-
.../doris/mtmv/ivm/agg/IvmAggArrayProcessor.java | 2 +-
.../doris/mtmv/ivm/agg/IvmAggAvgProcessor.java | 6 +
.../ivm/agg/IvmAggBitmapUnionCountProcessor.java | 6 +
.../mtmv/ivm/agg/IvmAggBitmapUnionProcessor.java | 2 +-
.../doris/mtmv/ivm/agg/IvmAggCountProcessor.java | 8 +-
.../mtmv/ivm/agg/IvmAggExtremalProcessor.java | 2 +-
.../mtmv/ivm/agg/IvmAggFunctionProcessor.java | 17 +
.../doris/mtmv/ivm/agg/IvmAggFunctionRegistry.java | 8 +
.../org/apache/doris/mtmv/ivm/agg/IvmAggMeta.java | 8 +
.../doris/mtmv/ivm/agg/IvmAggSumProcessor.java | 2 +-
.../apache/doris/mtmv/ivm/agg/IvmAggTarget.java | 45 +-
.../doris/mtmv/ivm/agg/IvmAggTargetSpec.java | 5 +-
.../nereids/rules/analysis/IvmNormalizeMTMV.java | 256 ++++-
.../mtmv/ivm/agg/IvmAggColumnSharingTest.java | 3 +-
.../mtmv/ivm/agg/IvmAggProcessorTestBase.java | 4 +-
.../rules/analysis/IvmNormalizeMTMVTest.java | 231 +++++
.../mtmv_p0/ivm/test_ivm_agg_expr_over_agg_1.out | 1023 ++++++++++++++++++++
.../mtmv_p0/ivm/test_ivm_agg_expr_over_agg_2.out | 279 ++++++
.../mtmv_p0/ivm/test_ivm_agg_expr_over_agg_3.out | 201 ++++
.../ivm/test_ivm_agg_expr_over_agg_1.groovy | 668 +++++++++++++
.../ivm/test_ivm_agg_expr_over_agg_2.groovy | 483 +++++++++
.../ivm/test_ivm_agg_expr_over_agg_3.groovy | 187 ++++
23 files changed, 3415 insertions(+), 33 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/AGENTS.md b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/AGENTS.md
index 41f75dc6f26..d980738a01c 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/AGENTS.md
+++ b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/AGENTS.md
@@ -191,7 +191,7 @@ The same `IvmUtil.buildRowIdHash()` function is used by both the normalize phase
## Backward Compatibility
-IVM is not publicly available until October 2026. Before that date, there is no need to maintain backward compatibility with existing IVM materialized views. Breaking changes to IVM metadata, DDL format, or internal storage layout are acceptable without migration support.
+IVM is not publicly available until January 2027. Before that date, there is no need to maintain backward compatibility with existing IVM materialized views, nor with the binlog/stream data they consume. Breaking changes to IVM metadata, DDL format, binlog format, or the internal storage layout are acceptable without migration support: a view created by an earlier build may be dropped and recreated instead of migrated, and a change that only affects such a view is not a correctness defec [...]
## Regression Test Guide: Binlog Operations
diff --git a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggArrayProcessor.java b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggArrayProcessor.java
index 269f298069d..3daeade76f2 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggArrayProcessor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggArrayProcessor.java
@@ -65,7 +65,7 @@ abstract class IvmAggArrayProcessor extends IvmAggFunctionProcessor {
@Override
public void appendApplyExpressions(IvmAggTarget target, IvmAggApplyContext applyContext) {
IvmAggExpressionBuilder ctx = applyContext.expressions();
- Slot oldArray = applyContext.rawMvSlot(target.getVisibleSlot().getName());
+ Slot oldArray = applyContext.rawMvSlot(target.getValueStateColumnName());
Expression empty = ctx.emptyArrayLiteral(target.getVisibleSlot().getDataType());
// The old MV side is genuinely NULL for groups that appear only in the delta (new groups).
// The ins/del transient columns can never be NULL -- every delta group holds at least one
diff --git a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggAvgProcessor.java b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggAvgProcessor.java
index f9cdb55f72c..acb2e47b112 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggAvgProcessor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggAvgProcessor.java
@@ -50,6 +50,12 @@ class IvmAggAvgProcessor extends IvmAggSumLikeProcessor {
return ImmutableList.of(IvmAggStateKey.SUM, IvmAggStateKey.COUNT);
}
+ @Override
+ boolean visibleColumnHoldsValueState(IvmAggTarget target) {
+ // AVG is derived from the hidden SUM/COUNT states; the visible AVG value is not merged.
+ return false;
+ }
+
@Override
public void appendApplyExpressions(IvmAggTarget target, IvmAggApplyContext applyContext) {
IvmAggExpressionBuilder ctx = applyContext.expressions();
diff --git a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggBitmapUnionCountProcessor.java b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggBitmapUnionCountProcessor.java
index 45a5be51eb9..0da77b432f2 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggBitmapUnionCountProcessor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggBitmapUnionCountProcessor.java
@@ -45,6 +45,12 @@ class IvmAggBitmapUnionCountProcessor extends IvmAggBitmapProcessor {
return ImmutableList.of(IvmAggStateKey.BITMAP_UNION);
}
+ @Override
+ boolean visibleColumnHoldsValueState(IvmAggTarget target) {
+ // The visible value is bitmap_count(hidden bitmap state), so the bitmap is the merged state.
+ return false;
+ }
+
@Override
public void appendApplyExpressions(IvmAggTarget target, IvmAggApplyContext applyContext) {
// Keep the bitmap union as hidden MV state; the visible BITMAP_UNION_COUNT value is bitmap_count(state).
diff --git a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggBitmapUnionProcessor.java b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggBitmapUnionProcessor.java
index f71e52e9382..7ccd8f8440b 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggBitmapUnionProcessor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggBitmapUnionProcessor.java
@@ -36,7 +36,7 @@ class IvmAggBitmapUnionProcessor extends IvmAggBitmapProcessor {
@Override
public void appendApplyExpressions(IvmAggTarget target, IvmAggApplyContext applyContext) {
// The visible MV column is the bitmap state itself, so merge old visible state with insert-delta bitmap.
- Slot oldBitmap = applyContext.rawMvSlot(target.getVisibleSlot().getName());
+ Slot oldBitmap = applyContext.rawMvSlot(target.getValueStateColumnName());
applyContext.putFinalExpression(target, target.getVisibleSlot().getName(),
buildGuardedNewBitmap(target, applyContext, oldBitmap));
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggCountProcessor.java b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggCountProcessor.java
index cc6eb3f29c3..d85a35e366f 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggCountProcessor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggCountProcessor.java
@@ -52,6 +52,12 @@ class IvmAggCountProcessor extends IvmAggFunctionProcessor {
: ImmutableList.of(function.child(0));
}
+ @Override
+ boolean visibleColumnHoldsValueState(IvmAggTarget target) {
+ // COUNT(*) rewrites the visible column from the group count instead of merging it.
+ return !target.isCountStar();
+ }
+
@Override
public void appendDeltaAggregateOutputs(IvmAggTarget target, Slot dmlFactorSlot,
List<NamedExpression> outputs, IvmAggExpressionBuilder ctx) {
@@ -92,7 +98,7 @@ class IvmAggCountProcessor extends IvmAggFunctionProcessor {
}
Expression newCount = ctx.assertNonNegative(new Add(
- applyContext.oldMvSlotZeroIfNull(target.getVisibleSlot().getName()),
+ applyContext.oldMvSlotZeroIfNull(target.getValueStateColumnName()),
applyContext.deltaSlotValue(target, IvmAggFunctionKind.COUNT)),
"negative count for " + target.getVisibleSlot().getName());
applyContext.putFinalExpression(target, target.getVisibleSlot().getName(),
diff --git a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggExtremalProcessor.java b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggExtremalProcessor.java
index 2522efc297e..3954792fd1e 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggExtremalProcessor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggExtremalProcessor.java
@@ -107,7 +107,7 @@ abstract class IvmAggExtremalProcessor extends IvmAggFunctionProcessor {
@Override
public void appendApplyExpressions(IvmAggTarget target, IvmAggApplyContext applyContext) {
IvmAggExpressionBuilder ctx = applyContext.expressions();
- Slot oldExtreme = applyContext.rawMvSlot(target.getVisibleSlot().getName());
+ Slot oldExtreme = applyContext.rawMvSlot(target.getValueStateColumnName());
Expression deltaInsert = applyContext.deltaSlotValue(target, functionKind);
Expression deltaDel = applyContext.deltaSlotValue(target, deleteSlotRef(target));
Expression newCount = applyContext.buildNewHiddenCount(target);
diff --git a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggFunctionProcessor.java b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggFunctionProcessor.java
index 8042b852286..2466f90b8a9 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggFunctionProcessor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggFunctionProcessor.java
@@ -124,6 +124,23 @@ public abstract class IvmAggFunctionProcessor {
return ImmutableList.of();
}
+ /**
+ * Returns true when the visible MV column carries this target's own aggregate value state.
+ *
+ * <p>Apply of such a target merges the old value read from the visible column (SUM, COUNT(expr),
+ * MIN/MAX, COLLECT_LIST/ARRAY_AGG, BITMAP_UNION). Returns false when the old value is derived
+ * from hidden state instead (AVG's SUM/COUNT, BITMAP_UNION_COUNT's bitmap) or from the group
+ * count (COUNT(*)), which are hidden-named columns and always survive to the MV schema.
+ *
+ * <p>The visible column is not guaranteed to survive: an upper expression may consume it without
+ * projecting it (for example {@code SELECT SUM(v) * 100}). Normalize keeps every slot apply reads
+ * alive by materializing a hidden carrier column for such targets
+ * ({@link IvmAggTarget#getValueStateColumnName()}).
+ */
+ boolean visibleColumnHoldsValueState(IvmAggTarget target) {
+ return true;
+ }
+
/** Original aggregate arguments stored in {@link IvmAggTarget}; COUNT(*) stores an empty list. */
protected List<Expression> targetArguments(AggregateFunction function) {
return ImmutableList.of(function.child(0));
diff --git a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggFunctionRegistry.java b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggFunctionRegistry.java
index f10a23c934f..812667b5cd6 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggFunctionRegistry.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggFunctionRegistry.java
@@ -131,6 +131,14 @@ public class IvmAggFunctionRegistry {
processorFor(target).appendApplyExpressions(target, ctx);
}
+ /**
+ * Returns true when this target's own value state is carried by the visible MV column, so normalize
+ * must keep that column alive (materializing a hidden carrier column when an upper expression drops it).
+ */
+ public boolean visibleColumnHoldsValueState(IvmAggTarget target) {
+ return processorFor(target).visibleColumnHoldsValueState(target);
+ }
+
private IvmAggFunctionProcessor processorFor(IvmAggTarget target) {
IvmAggFunctionProcessor processor = processorByKind.get(target.getFunctionKind());
if (processor == null) {
diff --git a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggMeta.java b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggMeta.java
index 9e59395aad0..759757d1916 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggMeta.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggMeta.java
@@ -65,6 +65,14 @@ public class IvmAggMeta {
return aggTargets;
}
+ /**
+ * Returns a copy with the given aggregate targets, used by normalize to rebind targets to the
+ * state columns it materialized for them after the aggregate output slots were resolved.
+ */
+ public IvmAggMeta withAggTargets(List<IvmAggTarget> newAggTargets) {
+ return new IvmAggMeta(scalarAgg, groupKeySlots, groupCountSlot, newAggTargets);
+ }
+
@Override
public String toString() {
return "IvmAggMeta{scalarAgg=" + scalarAgg
diff --git a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggSumProcessor.java b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggSumProcessor.java
index b08d289772d..c25f2c34ec1 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggSumProcessor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggSumProcessor.java
@@ -50,7 +50,7 @@ class IvmAggSumProcessor extends IvmAggSumLikeProcessor {
public void appendApplyExpressions(IvmAggTarget target, IvmAggApplyContext applyContext) {
IvmAggExpressionBuilder ctx = applyContext.expressions();
Expression newSum = new Add(
- applyContext.oldMvSlotZeroIfNull(target.getVisibleSlot().getName()),
+ applyContext.oldMvSlotZeroIfNull(target.getValueStateColumnName()),
applyContext.deltaSlotValue(target, IvmAggFunctionKind.SUM));
Expression newCount = applyContext.buildNewHiddenCount(target);
applyContext.putFinalExpression(target, target.getHiddenStateSlot(IvmAggStateKey.COUNT).getName(), newCount);
diff --git a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggTarget.java b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggTarget.java
index 6f266205b9a..6c658dade98 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggTarget.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggTarget.java
@@ -42,16 +42,29 @@ public class IvmAggTarget {
private final Slot visibleSlot;
// Persisted hidden state column slots. For example, an AVG target has hidden SUM and COUNT states.
private final Map<IvmAggStateKey, Slot> hiddenStateSlots;
+ // Column carrying this target's own aggregate value state when the visible column does not
+ // survive to the MV (see IvmAggFunctionProcessor#visibleColumnHoldsValueState). Null means the
+ // visible column carries it: either because it reaches the MV as a visible column, or because
+ // this target's old value is derived from hidden state instead (AVG, BITMAP_UNION_COUNT,
+ // COUNT(*)). Apply reads the old value through getValueStateColumnName().
+ private final Slot valueStateSlot;
+ // True when the MV stores this aggregate's visible output under a name the analyzer generates (an
+ // unnamed select item is renamed to __<function>_<index>), so apply cannot read the state back under
+ // the aggregate's own column name and the state needs a carrier column of its own.
+ private final boolean visibleColumnNameGenerated;
// the expression(s) from the base scan that feed this aggregate
// (empty for COUNT(*); may be Slot or compound Expression like v1+v2)
private final List<Expression> exprArgs;
public IvmAggTarget(int ordinal, IvmAggFunctionKind functionKind, Slot visibleSlot,
- Map<IvmAggStateKey, Slot> hiddenStateSlots, List<Expression> exprArgs) {
+ Map<IvmAggStateKey, Slot> hiddenStateSlots, Slot valueStateSlot, boolean visibleColumnNameGenerated,
+ List<Expression> exprArgs) {
this.ordinal = ordinal;
this.functionKind = Objects.requireNonNull(functionKind);
this.visibleSlot = Objects.requireNonNull(visibleSlot);
this.hiddenStateSlots = ImmutableMap.copyOf(hiddenStateSlots);
+ this.valueStateSlot = valueStateSlot;
+ this.visibleColumnNameGenerated = visibleColumnNameGenerated;
this.exprArgs = ImmutableList.copyOf(exprArgs);
}
@@ -80,6 +93,36 @@ public class IvmAggTarget {
return hiddenStateSlots.get(stateKey);
}
+ /** Whether the MV stores this target's visible output under an analyzer-generated column name. */
+ public boolean visibleColumnNameIsGenerated() {
+ return visibleColumnNameGenerated;
+ }
+
+ /** Column carrying this target's own value state, or null when the visible column carries it. */
+ public Slot getValueStateSlot() {
+ return valueStateSlot;
+ }
+
+ /**
+ * Name of the MV column apply reads this target's old value state from.
+ *
+ * <p>Processors whose apply merges the old value from the MV ({@code visibleColumnHoldsValueState})
+ * read through this name, so a target whose visible column was dropped in favor of a materialized
+ * hidden carrier resolves to that carrier instead.
+ */
+ public String getValueStateColumnName() {
+ return valueStateSlot != null ? valueStateSlot.getName() : visibleSlot.getName();
+ }
+
+ /** Rebinds this target to the state slots materialized for it by normalize. */
+ public IvmAggTarget withStateSlots(Slot newValueStateSlot, Map<IvmAggStateKey, Slot> newHiddenStateSlots) {
+ if (Objects.equals(valueStateSlot, newValueStateSlot) && hiddenStateSlots.equals(newHiddenStateSlots)) {
+ return this;
+ }
+ return new IvmAggTarget(ordinal, functionKind, visibleSlot, newHiddenStateSlots, newValueStateSlot,
+ visibleColumnNameGenerated, exprArgs);
+ }
+
/**
* Returns the canonical column name for the given state type.
*
diff --git a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggTargetSpec.java b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggTargetSpec.java
index 9516107877c..ccbbdd2ef93 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggTargetSpec.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/mtmv/ivm/agg/IvmAggTargetSpec.java
@@ -64,6 +64,9 @@ public class IvmAggTargetSpec {
for (Map.Entry<IvmAggStateKey, Alias> entry : hiddenStateAliases.entrySet()) {
hiddenSlots.put(entry.getKey(), entry.getValue().toSlot());
}
- return new IvmAggTarget(ordinal, functionKind, visibleAlias.toSlot(), hiddenSlots.build(), targetArguments);
+ // No value-state carrier yet: normalize materializes one only if an upper layer drops the
+ // visible column, which is not known while the spec is still being built.
+ return new IvmAggTarget(ordinal, functionKind, visibleAlias.toSlot(), hiddenSlots.build(), null,
+ visibleAlias.isNameFromChild(), targetArguments);
}
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/IvmNormalizeMTMV.java b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/IvmNormalizeMTMV.java
index 9ce790d31e5..867ade5e2ff 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/IvmNormalizeMTMV.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/IvmNormalizeMTMV.java
@@ -213,8 +213,11 @@ public class IvmNormalizeMTMV extends DefaultPlanRewriter<IvmNormalizeMTMV.Norma
private IvmRewriteResult rewriteResult;
private final IvmAggFunctionRegistry aggFunctionRegistry = IvmAggFunctionRegistry.INSTANCE;
- private StatementContext statementContext;
private boolean useFullKeys;
+ // The IVM rewrite context of this statement; normalize only runs when it is present, so it needs no
+ // presence checks of its own.
+ private IvmRewriteContext ivmRewriteContext;
+ private Set<TableNameInfo> excludedTriggerTables = ImmutableSet.of();
private final IdentityHashMap<Plan, List<Slot>> identityKeysByNode = new IdentityHashMap<>();
private int sinkKeyCounter;
private int unionIdxCounter;
@@ -222,9 +225,10 @@ public class IvmNormalizeMTMV extends DefaultPlanRewriter<IvmNormalizeMTMV.Norma
@Override
public Plan rewriteRoot(Plan plan, JobContext jobContext) {
- boolean enabledByIvmRewriteContext = jobContext.getCascadesContext().getStatementContext()
- .isIvmMTMVRewrite();
- if (!enabledByIvmRewriteContext) {
+ StatementContext statementContext = jobContext.getCascadesContext().getStatementContext();
+ ivmRewriteContext = statementContext.getIvmRewriteContext().orElse(null);
+ if (ivmRewriteContext == null) {
+ // Not an IVM rewrite: nothing to normalize.
return plan;
}
// Idempotency: if already normalized (e.g. rewritten plan re-entering), skip.
@@ -234,24 +238,25 @@ public class IvmNormalizeMTMV extends DefaultPlanRewriter<IvmNormalizeMTMV.Norma
}
rewriteResult.setNormalizeRewritten(true);
this.rewriteResult = rewriteResult;
- statementContext = jobContext.getCascadesContext().getStatementContext();
+ excludedTriggerTables = statementContext.getExcludedTriggerTables();
this.useFullKeys = resolveUseFullKeys();
Plan result = plan.accept(this, NormalizeContext.ROOT);
+ validateAggStateColumnsPersisted(result);
rewriteResult.setNormalizedPlan(result);
IvmPlanSignature planSignature = new IvmPlanSignatureGenerator().generate(result);
rewriteResult.setPlanSignature(planSignature);
- IvmRewriteContext.Mode mode = statementContext.getIvmRewriteContext().get().getMode();
+ IvmRewriteContext.Mode mode = ivmRewriteContext.getMode();
if (mode == IvmRewriteContext.Mode.INCREMENTAL) {
// Incremental refresh relies on the stored IVM layout: a drift in the normalized
// plan (row-id generation path) would produce an unmatchable delta. Check the
// signature as soon as normalization completes so every incremental path fails
// fast instead of attempting a delta rewrite against a stale layout baseline.
- validatePlanSignature(statementContext.getIvmRewriteContext().get().getMtmv(), rewriteResult);
+ validatePlanSignature(ivmRewriteContext.getMtmv(), rewriteResult);
}
if (mode == IvmRewriteContext.Mode.CREATE) {
LOG.info("IVM normalized plan, mtmvName={}, mode={}, inputRoot={}, plan={}, canonicalString={}, "
+ "signature={}",
- statementContext.getIvmRewriteContext().get().getMtmvName(), mode,
+ ivmRewriteContext.getMtmvName(), mode,
plan.getClass().getSimpleName(), result.treeString(), planSignature.getCanonicalString(),
planSignature.getSha256());
}
@@ -278,15 +283,13 @@ public class IvmNormalizeMTMV extends DefaultPlanRewriter<IvmNormalizeMTMV.Norma
}
private boolean resolveUseFullKeys() {
- if (statementContext == null || !statementContext.getIvmRewriteContext().isPresent()) {
- return false;
- }
- IvmRewriteContext rewriteContext = statementContext.getIvmRewriteContext().get();
- if (rewriteContext.getUseFullKeys() != null) {
- return rewriteContext.getUseFullKeys();
+ if (ivmRewriteContext.getUseFullKeys() != null) {
+ return ivmRewriteContext.getUseFullKeys();
}
- if (rewriteContext.getMtmv() != null && rewriteContext.getMtmv().getIvmInfo() != null) {
- return rewriteContext.getMtmv().getIvmInfo().isUseFullKeys();
+ // CREATE has no MV yet, so its full-keys setting can only come from the context.
+ MTMV mtmv = ivmRewriteContext.getMtmv();
+ if (mtmv != null && mtmv.getIvmInfo() != null) {
+ return mtmv.getIvmInfo().isUseFullKeys();
}
return false;
}
@@ -369,8 +372,8 @@ public class IvmNormalizeMTMV extends DefaultPlanRewriter<IvmNormalizeMTMV.Norma
"IVM does not support DISTINCT project.");
}
Plan newChild = project.child().accept(this, context);
- List<NamedExpression> baseOutputs = rewriteOutputsWithIvmHiddenColumns(newChild, project.getProjects(),
- context.isFirstNonSink);
+ List<NamedExpression> baseOutputs = materializeDroppedAggState(
+ rewriteOutputsWithIvmHiddenColumns(newChild, project.getProjects(), context.isFirstNonSink));
List<Slot> childKeys = useFullKeys && !context.isInsideAggregate
? identityKeysByNode.get(newChild) : null;
@@ -935,7 +938,8 @@ public class IvmNormalizeMTMV extends DefaultPlanRewriter<IvmNormalizeMTMV.Norma
}
resolved.add(new IvmAggTarget(target.getOrdinal(), target.getFunctionKind(),
- resolvedVisible, resolvedHidden.build(), target.getExprArgs()));
+ resolvedVisible, resolvedHidden.build(), target.getValueStateSlot(),
+ target.visibleColumnNameIsGenerated(), target.getExprArgs()));
}
return resolved;
}
@@ -945,8 +949,8 @@ public class IvmNormalizeMTMV extends DefaultPlanRewriter<IvmNormalizeMTMV.Norma
public Plan visitLogicalResultSink(LogicalResultSink<? extends Plan> sink, NormalizeContext context) {
validateUserOutputColumnNames(sink.getOutputExprs());
Plan newChild = sink.child().accept(this, context);
- List<NamedExpression> baseOutputs = rewriteOutputsWithIvmHiddenColumns(newChild, sink.getOutputExprs(),
- context.isFirstNonSink);
+ List<NamedExpression> baseOutputs = materializeDroppedAggState(
+ rewriteOutputsWithIvmHiddenColumns(newChild, sink.getOutputExprs(), context.isFirstNonSink));
List<Slot> childKeys = useFullKeys ? identityKeysByNode.get(newChild) : null;
List<Slot> sinkKeys = new ArrayList<>();
@@ -1069,6 +1073,212 @@ public class IvmNormalizeMTMV extends DefaultPlanRewriter<IvmNormalizeMTMV.Norma
.collect(Collectors.toMap(Slot::getName, slot -> slot, (left, right) -> left, LinkedHashMap::new));
}
+ /**
+ * Materializes the aggregate state columns this layer drops, so an incremental refresh can still
+ * read the old aggregate state from the MV.
+ *
+ * <p>Every state slot the apply stage reads must be a persisted MV column, because apply resolves
+ * the old state by column name from the MV physical table. Hidden state columns are hidden-named
+ * and therefore propagate through every layer, but the aggregate functions whose own value is
+ * their mergeable state (SUM, COUNT(expr), MIN/MAX, COLLECT_LIST/ARRAY_AGG, BITMAP_UNION) keep
+ * that state in their visible column, which disappears as soon as an upper layer consumes it
+ * inside an expression without projecting it, as in {@code SELECT SUM(v) * 100}. Such a slot is
+ * materialized here as a bare pass-through hidden alias, and every target reading it is rebound
+ * to that alias.
+ *
+ * <p>Rebinding matters as much as materializing: the column pool lets a target reuse a visible
+ * aggregate column as its own hidden state (AVG reusing a visible SUM column), so a reusing target
+ * must follow the column's owner onto the materialized carrier instead of reading a column that no
+ * longer reaches the MV.
+ *
+ * <p>The materialized name is the name the delta sub-plan already generates for that state
+ * ({@link IvmUtil#ivmAggHiddenColumnName}, keyed by the owning target's ordinal and kind), so delta
+ * aggregate outputs and delta slot lookups are unaffected and both sides of the merge agree on the
+ * column name. A slot an earlier layer already materialized, or that another target already
+ * materialized for the same aggregate state, is reused instead of materializing a duplicate.
+ */
+ private List<NamedExpression> materializeDroppedAggState(List<NamedExpression> outputs) {
+ IvmAggMeta aggMeta = rewriteResult.getAggMeta();
+ if (aggMeta == null) {
+ // Below the aggregate no target is known yet, so no aggregate state can be dropped here.
+ return outputs;
+ }
+ List<NamedExpression> extendedOutputs = new ArrayList<>(outputs);
+ List<IvmAggTarget> reboundTargets = new ArrayList<>(aggMeta.getAggTargets().size());
+ boolean rebound = false;
+ // While the layout is being created there is no MV yet (CREATE MATERIALIZED VIEW keeps only its
+ // name), so normalize decides the layout itself; a refresh is bound by the MV's own schema.
+ MTMV mtmv = ivmRewriteContext.getMtmv();
+ for (IvmAggTarget target : aggMeta.getAggTargets()) {
+ Slot valueStateSlot = valueStateSlotFor(target, mtmv, extendedOutputs, aggMeta);
+ ImmutableMap.Builder<IvmAggStateKey, Slot> hiddenStateSlots = ImmutableMap.builder();
+ for (Map.Entry<IvmAggStateKey, Slot> hiddenStateSlot : target.getHiddenStateSlots().entrySet()) {
+ hiddenStateSlots.put(hiddenStateSlot.getKey(),
+ materializeAggStateSlot(extendedOutputs, hiddenStateSlot.getValue(), aggMeta));
+ }
+ IvmAggTarget reboundTarget = target.withStateSlots(valueStateSlot, hiddenStateSlots.build());
+ rebound |= reboundTarget != target;
+ reboundTargets.add(reboundTarget);
+ }
+ if (rebound) {
+ // Keep the rebinding visible to the layers above: a layer that passes a state column through
+ // under a different slot (the refresh sink rebinds the normalized hidden columns to the MV's
+ // own slots) changes which slot carries the state without adding any column.
+ rewriteResult.setAggMeta(aggMeta.withAggTargets(reboundTargets));
+ }
+ return extendedOutputs;
+ }
+
+ /**
+ * Returns the slot this target's own value state is read from, materializing a carrier column when the
+ * state needs a column the layout does not have yet.
+ *
+ * <p>When {@code mtmv} is null the layout is being created (CREATE MATERIALIZED VIEW knows only its
+ * name): the state is the visible aggregate column when that column survives into the MV, and a
+ * materialized carrier otherwise. On a refresh {@code mtmv} exists and its schema owns the layout: the
+ * state is the carrier only when the MV really has that column, and otherwise the MV's visible column
+ * carries it. That keeps the refresh path independent of the binder's insert projects, which rename a
+ * state column locally and rebuild it as the MV column under its own name, so a user column that
+ * happens to be named like the aggregate's generated alias never becomes the state.
+ */
+ private Slot valueStateSlotFor(IvmAggTarget target, MTMV mtmv, List<NamedExpression> outputs,
+ IvmAggMeta aggMeta) {
+ if (!aggFunctionRegistry.visibleColumnHoldsValueState(target)) {
+ // AVG, BITMAP_UNION_COUNT and COUNT(*) merge hidden state or the group count instead.
+ return target.getValueStateSlot();
+ }
+ // The column currently carrying this target's value: the carrier materialized by a lower layer if
+ // there is one, otherwise the visible column the aggregate still produces.
+ Slot stateSlot = target.getValueStateSlot() != null
+ ? target.getValueStateSlot() : target.getVisibleSlot();
+ if (mtmv == null) {
+ // Otherwise the projection decides: the state keeps the column the plan projects it as (the
+ // aggregate output itself or a rename of it), and a state the projection drops gets a carrier.
+ return materializeAggStateSlot(outputs, stateSlot, aggMeta);
+ }
+ String carrierName = IvmUtil.ivmAggHiddenColumnName(target.getOrdinal(),
+ target.getFunctionKind().name());
+ if (mtmv.getColumn(carrierName) != null) {
+ return materializeAggStateSlot(outputs, stateSlot, aggMeta);
+ }
+ // Without a carrier the MV keeps the state in the column the plan projected it as. That name is
+ // only usable when the MV really has it: the binder renames a state column to the MV's own column
+ // name for an unnamed aggregate (COUNT(*) becomes __count_0), while for a clamped key column it
+ // renames the state to a project-local name and rebuilds the MV column by coercion, in which case
+ // the MV's visible column carries the state.
+ NamedExpression projected = findProjectedKey(outputs, stateSlot);
+ return projected != null && mtmv.getColumn(projected.getName()) != null
+ ? projected.toSlot() : null;
+ }
+
+ /**
+ * Returns the slot that carries {@code stateSlot} above this layer, appending a hidden pass-through
+ * alias when this layer drops it.
+ *
+ * <p>A layer keeps the state alive when it projects the state slot itself or a pure rename of it, which
+ * is how the refresh sink maps the normalized hidden columns onto the MV's own slots. Only when the
+ * state slot is really gone from the layer's output does it need a carrier.
+ */
+ private Slot materializeAggStateSlot(List<NamedExpression> outputs, Slot stateSlot, IvmAggMeta aggMeta) {
+ NamedExpression projected = findProjectedKey(outputs, stateSlot);
+ if (projected != null) {
+ // The projecting output's own slot is what carries the value: the column itself, a column it
+ // projects through, or a carrier another target needed for this same state.
+ return projected.toSlot();
+ }
+ return materializeStateCarrier(outputs, stateSlot, aggMeta);
+ }
+
+ /**
+ * Appends a hidden carrier column for {@code stateSlot} and returns its slot.
+ *
+ * <p>The carrier is named after the owning target's ordinal and kind, which is the name the delta
+ * sub-plan already generates for the same state, so no delta-side lookup changes.
+ */
+ private Slot materializeStateCarrier(List<NamedExpression> outputs, Slot stateSlot, IvmAggMeta aggMeta) {
+ IvmAggTarget owner = aggTargetOwningVisibleSlot(stateSlot, aggMeta);
+ Alias carrier = new Alias(stateSlot,
+ IvmUtil.ivmAggHiddenColumnName(owner.getOrdinal(), owner.getFunctionKind().name()));
+ outputs.add(carrier);
+ return carrier.toSlot();
+ }
+
+ /**
+ * Finds the target whose visible column is {@code stateSlot}. That target owns the state column's
+ * name: the carrier is named after its ordinal and kind so the name matches the delta sub-plan's
+ * name for the same aggregate state.
+ */
+ private IvmAggTarget aggTargetOwningVisibleSlot(Slot stateSlot, IvmAggMeta aggMeta) {
+ for (IvmAggTarget target : aggMeta.getAggTargets()) {
+ if (target.getVisibleSlot().getExprId().equals(stateSlot.getExprId())) {
+ return target;
+ }
+ }
+ throw new IvmException(IvmFailureReason.PLAN_REWRITE_FAILED,
+ "IVM normalization error: aggregate state column '" + stateSlot.getName()
+ + "' is dropped by an upper projection but no visible aggregate output owns it");
+ }
+
+ /**
+ * Checks that every aggregate state slot the apply stage reads survived normalization into the MV
+ * schema.
+ *
+ * <p>Apply reads the old state by column name from the MV physical table, so a state column missing
+ * from the normalized output would surface only much later, as a PLAN_REWRITE_FAILED during the
+ * first incremental refresh of an otherwise accepted MV. Failing here instead keeps the failure
+ * attached to the statement that produced the layout: CREATE MATERIALIZED VIEW, or a refresh
+ * re-normalizing the same query SQL.
+ *
+ * <p>Targets whose visible column the analyzer stores under a generated name (an unnamed aggregate
+ * such as {@code SUM(v1)} is stored as {@code __sum_1}) are skipped: their state column name cannot
+ * be mapped from the plan at all, so their incremental refresh was never maintainable and the check
+ * has nothing to compare against. Their hidden state columns are still checked.
+ */
+ private void validateAggStateColumnsPersisted(Plan normalizedPlan) {
+ IvmAggMeta aggMeta = rewriteResult.getAggMeta();
+ if (aggMeta == null) {
+ return;
+ }
+ // The layout is only defined where the view is created or where an incremental refresh reads the
+ // old state, and those are the two places this check runs: at CREATE it reports early that the
+ // layout being defined cannot be maintained incrementally, and on an incremental refresh it
+ // reports the state a merge would read. NORMALIZE and FULL only re-analyze the layout of an
+ // existing view, where a state column can be absent for reasons this check cannot judge -- the
+ // view then still completes a full refresh, which is how such a layout is rebuilt.
+ IvmRewriteContext.Mode mode = ivmRewriteContext.getMode();
+ if (mode != IvmRewriteContext.Mode.CREATE && mode != IvmRewriteContext.Mode.INCREMENTAL) {
+ return;
+ }
+ Set<String> persistedColumns = normalizedPlan.getOutput().stream()
+ .map(Slot::getName)
+ .collect(Collectors.toSet());
+ List<Slot> requiredStateSlots = new ArrayList<>();
+ requiredStateSlots.add(aggMeta.getGroupCountSlot());
+ // A visible column the analyzer stores under a generated name (an unnamed aggregate such as
+ // COUNT(v1), or a hidden state column the pool shared with it) cannot be checked by name: the MV
+ // renames it after normalize, so its incremental refresh was never maintainable.
+ Set<String> generatedColumnNames = aggMeta.getAggTargets().stream()
+ .filter(IvmAggTarget::visibleColumnNameIsGenerated)
+ .map(target -> target.getVisibleSlot().getName())
+ .collect(Collectors.toSet());
+ for (IvmAggTarget target : aggMeta.getAggTargets()) {
+ if (aggFunctionRegistry.visibleColumnHoldsValueState(target)) {
+ requiredStateSlots.add(target.getValueStateSlot() != null
+ ? target.getValueStateSlot() : target.getVisibleSlot());
+ }
+ requiredStateSlots.addAll(target.getHiddenStateSlots().values());
+ }
+ for (Slot stateSlot : requiredStateSlots) {
+ if (!persistedColumns.contains(stateSlot.getName())
+ && !generatedColumnNames.contains(stateSlot.getName())) {
+ throw new IvmException(IvmFailureReason.PLAN_REWRITE_FAILED,
+ "IVM normalization error: aggregate state column '" + stateSlot.getName()
+ + "' is missing from the normalized MV output, so incremental refresh"
+ + " could not read the old aggregate state");
+ }
+ }
+ }
+
private NamedExpression rewriteIvmHiddenOutput(NamedExpression output, Map<String, Slot> ivmHiddenSlotsByName) {
Slot ivmHiddenSlot = ivmHiddenSlotsByName.get(output.getName());
if (ivmHiddenSlot == null) {
@@ -1224,7 +1434,7 @@ public class IvmNormalizeMTMV extends DefaultPlanRewriter<IvmNormalizeMTMV.Norma
}
private boolean isExcludedTriggerTable(LogicalOlapScan scan) {
- if (statementContext == null || statementContext.getExcludedTriggerTables().isEmpty()) {
+ if (excludedTriggerTables.isEmpty()) {
return false;
}
OlapTable table = scan.getTable();
@@ -1239,7 +1449,7 @@ public class IvmNormalizeMTMV extends DefaultPlanRewriter<IvmNormalizeMTMV.Norma
: InternalCatalog.INTERNAL_CATALOG_NAME;
tableNameInfo = new TableNameInfo(ctlName, dbName, table.getName());
}
- return MTMVPartitionUtil.isTableExcluded(statementContext.getExcludedTriggerTables(), tableNameInfo);
+ return MTMVPartitionUtil.isTableExcluded(excludedTriggerTables, tableNameInfo);
}
}
diff --git a/fe/fe-core/src/test/java/org/apache/doris/mtmv/ivm/agg/IvmAggColumnSharingTest.java b/fe/fe-core/src/test/java/org/apache/doris/mtmv/ivm/agg/IvmAggColumnSharingTest.java
index ff0993dfa8a..5c19d3020c6 100644
--- a/fe/fe-core/src/test/java/org/apache/doris/mtmv/ivm/agg/IvmAggColumnSharingTest.java
+++ b/fe/fe-core/src/test/java/org/apache/doris/mtmv/ivm/agg/IvmAggColumnSharingTest.java
@@ -171,6 +171,7 @@ class IvmAggColumnSharingTest extends IvmAggProcessorTestBase {
Map<IvmAggStateKey, Slot> hidden = new LinkedHashMap<>(target.getHiddenStateSlots());
hidden.put(key, slot);
return new IvmAggTarget(target.getOrdinal(), target.getFunctionKind(),
- target.getVisibleSlot(), hidden, target.getExprArgs());
+ target.getVisibleSlot(), hidden, target.getValueStateSlot(),
+ target.visibleColumnNameIsGenerated(), target.getExprArgs());
}
}
diff --git a/fe/fe-core/src/test/java/org/apache/doris/mtmv/ivm/agg/IvmAggProcessorTestBase.java b/fe/fe-core/src/test/java/org/apache/doris/mtmv/ivm/agg/IvmAggProcessorTestBase.java
index 7e3e5ccc347..788202d8c04 100644
--- a/fe/fe-core/src/test/java/org/apache/doris/mtmv/ivm/agg/IvmAggProcessorTestBase.java
+++ b/fe/fe-core/src/test/java/org/apache/doris/mtmv/ivm/agg/IvmAggProcessorTestBase.java
@@ -89,7 +89,9 @@ abstract class IvmAggProcessorTestBase {
protected IvmAggTarget target(int ordinal, IvmAggFunctionKind kind, String visibleName,
DataType visibleType, Map<IvmAggStateKey, Slot> hiddenSlots, List<Expression> args) {
- return new IvmAggTarget(ordinal, kind, slot(visibleName, visibleType), hiddenSlots, args);
+ // No value-state carrier, and the visible column keeps its own name: these targets keep their
+ // value in the visible column.
+ return new IvmAggTarget(ordinal, kind, slot(visibleName, visibleType), hiddenSlots, null, false, args);
}
protected Slot slot(String name, DataType dataType) {
diff --git a/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/analysis/IvmNormalizeMTMVTest.java b/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/analysis/IvmNormalizeMTMVTest.java
index a64a223960c..e20d7be3ede 100644
--- a/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/analysis/IvmNormalizeMTMVTest.java
+++ b/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/analysis/IvmNormalizeMTMVTest.java
@@ -30,6 +30,7 @@ import org.apache.doris.catalog.PartitionInfo;
import org.apache.doris.catalog.TableProperty;
import org.apache.doris.catalog.Type;
import org.apache.doris.catalog.info.TableNameInfo;
+import org.apache.doris.common.Pair;
import org.apache.doris.mtmv.ivm.IvmException;
import org.apache.doris.mtmv.ivm.IvmFailureReason;
import org.apache.doris.mtmv.ivm.IvmInfo;
@@ -53,10 +54,12 @@ import org.apache.doris.nereids.trees.expressions.Expression;
import org.apache.doris.nereids.trees.expressions.NamedExpression;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.expressions.SlotReference;
+import org.apache.doris.nereids.trees.expressions.functions.agg.AggregateFunction;
import org.apache.doris.nereids.trees.expressions.functions.agg.AnyValue;
import org.apache.doris.nereids.trees.expressions.functions.agg.Avg;
import org.apache.doris.nereids.trees.expressions.functions.agg.BitmapUnion;
import org.apache.doris.nereids.trees.expressions.functions.agg.BitmapUnionCount;
+import org.apache.doris.nereids.trees.expressions.functions.agg.CollectList;
import org.apache.doris.nereids.trees.expressions.functions.agg.Count;
import org.apache.doris.nereids.trees.expressions.functions.agg.Max;
import org.apache.doris.nereids.trees.expressions.functions.agg.Min;
@@ -65,6 +68,7 @@ import org.apache.doris.nereids.trees.expressions.functions.scalar.BitmapFromStr
import org.apache.doris.nereids.trees.expressions.functions.scalar.MurmurHash3128;
import org.apache.doris.nereids.trees.expressions.functions.scalar.UuidNumeric;
import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.IntegerLiteral;
import org.apache.doris.nereids.trees.expressions.literal.LargeIntLiteral;
import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
import org.apache.doris.nereids.trees.expressions.literal.TinyIntLiteral;
@@ -653,6 +657,233 @@ class IvmNormalizeMTMVTest {
Assertions.assertTrue(rewriteResult.isDeterministic(rowIdAlias.toSlot()));
}
+ /**
+ * Builds {@code Project(id, f(aggOutput)) → Aggregate(id, aggOutput)}: the aggregate output only
+ * feeds an upper expression, which is the shape that made the visible aggregate column disappear
+ * from the MV ({@code SELECT id, SUM(name) * 100}).
+ */
+ private LogicalProject<Plan> buildProjectedAggExpr(Alias aggAlias, Alias secondAggAlias,
+ Alias exprAlias) {
+ Slot idSlot = scan.getOutput().get(0);
+ List<NamedExpression> aggOutputs = secondAggAlias == null
+ ? ImmutableList.of(idSlot, aggAlias)
+ : ImmutableList.of(idSlot, aggAlias, secondAggAlias);
+ LogicalAggregate<Plan> agg = new LogicalAggregate<>(
+ ImmutableList.of(idSlot), aggOutputs, true, java.util.Optional.empty(), scan);
+ return new LogicalProject<>(ImmutableList.of(idSlot, exprAlias), agg);
+ }
+
+ @Test
+ void testDroppedVisibleAggColumnGetsMaterializedStateCarrier() {
+ Slot nameSlot = scan.getOutput().get(1);
+ // SELECT id, SUM(name) * 100 AS s100 FROM t GROUP BY id
+ Alias sumAlias = new Alias(new Sum(nameSlot), "sum(name)");
+ Alias s100 = new Alias(new org.apache.doris.nereids.trees.expressions.Multiply(
+ sumAlias.toSlot(), new IntegerLiteral(100)), "s100");
+ LogicalProject<Plan> project = buildProjectedAggExpr(sumAlias, null, s100);
+
+ JobContext jobContext = newJobContextForRoot(project, true);
+ Plan result = new IvmNormalizeMTMV().rewriteRoot(project, jobContext);
+
+ String carrierName = IvmUtil.ivmAggHiddenColumnName(0, "SUM");
+ List<String> outputNames = result.getOutput().stream()
+ .map(Slot::getName).collect(Collectors.toList());
+ Assertions.assertTrue(outputNames.contains(carrierName), "outputs=" + outputNames);
+ Assertions.assertFalse(outputNames.contains(sumAlias.toSlot().getName()));
+
+ IvmAggMeta aggMeta = jobContext.getCascadesContext().getIvmRewriteResult().get().getAggMeta();
+ Assertions.assertEquals(1, aggMeta.getAggTargets().size());
+ IvmAggTarget target = aggMeta.getAggTargets().get(0);
+ Assertions.assertEquals(carrierName, target.getValueStateColumnName());
+ Assertions.assertNotNull(target.getValueStateSlot());
+ Assertions.assertFalse(target.getHiddenStateSlots().containsKey(IvmAggStateKey.SUM));
+ Assertions.assertNotNull(target.getHiddenStateSlot(IvmAggStateKey.COUNT));
+
+ // The carrier is a bare pass-through of the aggregate's visible output, so the merge result
+ // apply computes for that column is exactly the state the MV stores.
+ Alias carrier = ((LogicalProject<?>) result).getProjects().stream()
+ .filter(Alias.class::isInstance)
+ .map(Alias.class::cast)
+ .filter(alias -> carrierName.equals(alias.getName()))
+ .findFirst()
+ .get();
+ Assertions.assertEquals(sumAlias.toSlot(), carrier.child());
+ }
+
+ @Test
+ void testProjectedVisibleAggColumnKeepsCarryingItsOwnState() {
+ Slot idSlot = scan.getOutput().get(0);
+ Slot nameSlot = scan.getOutput().get(1);
+ // SELECT id, SUM(name) AS s, SUM(name) * 100 AS s100 FROM t GROUP BY id: the visible SUM column
+ // is projected, so it carries the state itself and no hidden SUM carrier is added.
+ Alias sumAlias = new Alias(new Sum(nameSlot), "s");
+ Alias s100 = new Alias(new org.apache.doris.nereids.trees.expressions.Multiply(
+ sumAlias.toSlot(), new IntegerLiteral(100)), "s100");
+ LogicalProject<Plan> project = buildProjectedAggExpr(sumAlias, null, s100)
+ .withProjects(ImmutableList.of(idSlot, sumAlias.toSlot(), s100));
+
+ JobContext jobContext = newJobContextForRoot(project, true);
+ Plan result = new IvmNormalizeMTMV().rewriteRoot(project, jobContext);
+
+ List<String> outputNames = result.getOutput().stream()
+ .map(Slot::getName).collect(Collectors.toList());
+ Assertions.assertFalse(outputNames.contains(IvmUtil.ivmAggHiddenColumnName(0, "SUM")),
+ "outputs=" + outputNames);
+
+ IvmAggMeta aggMeta = jobContext.getCascadesContext().getIvmRewriteResult().get().getAggMeta();
+ IvmAggTarget target = aggMeta.getAggTargets().get(0);
+ // The projected column is the state column itself, so apply reads it under the same name.
+ Assertions.assertEquals("s", target.getValueStateColumnName());
+ }
+
+ @Test
+ void testReusedHiddenStateFollowsDroppedVisibleColumnOntoItsCarrier() {
+ Slot idSlot = scan.getOutput().get(0);
+ Slot nameSlot = scan.getOutput().get(1);
+ // SELECT id, SUM(name) * 100 AS s100, AVG(name) * 2 AS a2 FROM t GROUP BY id: AVG's hidden SUM
+ // state reuses the visible SUM column (column pool), which the upper expressions then drop, so
+ // AVG must follow that column onto the single materialized carrier.
+ Alias sumAlias = new Alias(new Sum(nameSlot), "sum(name)");
+ Alias avgAlias = new Alias(new Avg(nameSlot), "avg(name)");
+ Alias s100 = new Alias(new org.apache.doris.nereids.trees.expressions.Multiply(
+ sumAlias.toSlot(), new IntegerLiteral(100)), "s100");
+ Alias a2 = new Alias(new org.apache.doris.nereids.trees.expressions.Multiply(
+ avgAlias.toSlot(), new IntegerLiteral(2)), "a2");
+ LogicalProject<Plan> project = buildProjectedAggExpr(sumAlias, avgAlias, s100);
+ project = project.withProjects(ImmutableList.of(idSlot, s100, a2));
+
+ JobContext jobContext = newJobContextForRoot(project, true);
+ Plan result = new IvmNormalizeMTMV().rewriteRoot(project, jobContext);
+
+ IvmAggMeta aggMeta = jobContext.getCascadesContext().getIvmRewriteResult().get().getAggMeta();
+ Assertions.assertEquals(2, aggMeta.getAggTargets().size());
+ IvmAggTarget sumTarget = aggMeta.getAggTargets().get(0);
+ IvmAggTarget avgTarget = aggMeta.getAggTargets().get(1);
+ Assertions.assertEquals(IvmUtil.ivmAggHiddenColumnName(0, "SUM"), sumTarget.getValueStateColumnName());
+ Assertions.assertEquals(sumTarget.getValueStateSlot().getExprId(),
+ avgTarget.getHiddenStateSlot(IvmAggStateKey.SUM).getExprId());
+ // AVG's hidden COUNT still shares the SUM target's hidden COUNT column.
+ Assertions.assertEquals(sumTarget.getHiddenStateSlot(IvmAggStateKey.COUNT).getExprId(),
+ avgTarget.getHiddenStateSlot(IvmAggStateKey.COUNT).getExprId());
+
+ // Both targets share one carrier column, so the MV gains exactly one column for the SUM state.
+ Assertions.assertEquals(1, result.getOutput().stream()
+ .filter(slot -> slot.getName().equals(IvmUtil.ivmAggHiddenColumnName(0, "SUM")))
+ .count());
+ }
+
+ /** Hidden aggregate columns of the normalized plan output, in plan order. */
+ private List<String> hiddenAggColumnNames(Plan normalizedPlan) {
+ String aggPrefix = Column.IVM_HIDDEN_COLUMN_PREFIX + "AGG_";
+ return normalizedPlan.getOutput().stream()
+ .map(Slot::getName)
+ .filter(name -> name.startsWith(aggPrefix))
+ .collect(Collectors.toList());
+ }
+
+ @Test
+ void testWrappedAggStateCarriersPerFunction() {
+ Slot idSlot = scan.getOutput().get(0);
+ Slot nameSlot = scan.getOutput().get(1);
+ List<String> groupCount = ImmutableList.of(Column.IVM_AGG_COUNT_COL);
+
+ // Every wrapped shape materializes exactly the state its apply stage merges: the column
+ // carrying the aggregate's own value when the upper expression drops the visible column,
+ // and always the hidden state columns (a hidden state column is hidden-named, so it reaches
+ // the MV without help). AVG, BITMAP_UNION_COUNT and COUNT(*) derive the visible value from
+ // hidden state or from the group count, so wrapping them adds nothing.
+ List<Pair<AggregateFunction, List<String>>> shapes = ImmutableList.of(
+ Pair.of(new Sum(nameSlot), concat(groupCount,
+ IvmUtil.ivmAggHiddenColumnName(0, "COUNT"), IvmUtil.ivmAggHiddenColumnName(0, "SUM"))),
+ Pair.of(new Count(nameSlot), concat(groupCount,
+ IvmUtil.ivmAggHiddenColumnName(0, "COUNT"))),
+ Pair.of(new Min(nameSlot), concat(groupCount,
+ IvmUtil.ivmAggHiddenColumnName(0, "COUNT"), IvmUtil.ivmAggHiddenColumnName(0, "MIN"))),
+ Pair.of(new Max(nameSlot), concat(groupCount,
+ IvmUtil.ivmAggHiddenColumnName(0, "COUNT"), IvmUtil.ivmAggHiddenColumnName(0, "MAX"))),
+ Pair.of(new CollectList(nameSlot), concat(groupCount,
+ IvmUtil.ivmAggHiddenColumnName(0, "COLLECT_LIST"))),
+ Pair.of(new BitmapUnion(new BitmapFromString(nameSlot)), concat(groupCount,
+ IvmUtil.ivmAggHiddenColumnName(0, "BITMAP_UNION"))),
+ Pair.of(new Avg(nameSlot), concat(groupCount,
+ IvmUtil.ivmAggHiddenColumnName(0, "SUM"), IvmUtil.ivmAggHiddenColumnName(0, "COUNT"))),
+ Pair.of(new BitmapUnionCount(new BitmapFromString(nameSlot)), concat(groupCount,
+ IvmUtil.ivmAggHiddenColumnName(0, "BITMAP_UNION"))),
+ Pair.of(new Count(), groupCount));
+
+ for (Pair<AggregateFunction, List<String>> shape : shapes) {
+ AggregateFunction function = shape.first;
+ Alias aggAlias = new Alias(function, "wrapped");
+ Alias exprAlias = new Alias(new org.apache.doris.nereids.trees.expressions.Multiply(
+ aggAlias.toSlot(), new IntegerLiteral(100)), "out");
+ LogicalAggregate<Plan> agg = new LogicalAggregate<>(
+ ImmutableList.of(idSlot), ImmutableList.of(idSlot, aggAlias), true,
+ java.util.Optional.empty(), scan);
+ LogicalProject<Plan> project = new LogicalProject<>(ImmutableList.of(idSlot, exprAlias), agg);
+
+ Plan result = new IvmNormalizeMTMV().rewriteRoot(project, newJobContextForRoot(project, true));
+ Assertions.assertEquals(shape.second, hiddenAggColumnNames(result),
+ "unexpected hidden layout for wrapped " + function.getName());
+ }
+ }
+
+ private List<String> concat(List<String> head, String... tail) {
+ return ImmutableList.<String>builder().addAll(head).add(tail).build();
+ }
+
+ @Test
+ void testWrappedScalarAggMaterializesStateCarrier() {
+ Slot nameSlot = scan.getOutput().get(1);
+ // SELECT SUM(name) * 100 FROM t — no GROUP BY.
+ Alias sumAlias = new Alias(new Sum(nameSlot), "sum(name)");
+ LogicalAggregate<Plan> agg = new LogicalAggregate<>(
+ ImmutableList.of(), ImmutableList.of(sumAlias), true, java.util.Optional.empty(), scan);
+ Alias s100 = new Alias(new org.apache.doris.nereids.trees.expressions.Multiply(
+ sumAlias.toSlot(), new IntegerLiteral(100)), "s100");
+ LogicalProject<Plan> project = new LogicalProject<>(ImmutableList.of(s100), agg);
+
+ Plan result = new IvmNormalizeMTMV().rewriteRoot(project, newJobContextForRoot(project, true));
+
+ Assertions.assertEquals(ImmutableList.of(Column.IVM_AGG_COUNT_COL,
+ IvmUtil.ivmAggHiddenColumnName(0, "COUNT"),
+ IvmUtil.ivmAggHiddenColumnName(0, "SUM")),
+ hiddenAggColumnNames(result));
+ }
+
+ @Test
+ void testRefreshLayoutComesFromTheMvSchema() {
+ Slot idSlot = scan.getOutput().get(0);
+ Slot nameSlot = scan.getOutput().get(1);
+ // SELECT id, SUM(name) * 100 AS s100 FROM t GROUP BY id, refreshed as an existing MV.
+ Alias sumAlias = new Alias(new Sum(nameSlot), "sum(name)");
+ Alias s100 = new Alias(new org.apache.doris.nereids.trees.expressions.Multiply(
+ sumAlias.toSlot(), new IntegerLiteral(100)), "s100");
+ LogicalAggregate<Plan> agg = new LogicalAggregate<>(
+ ImmutableList.of(idSlot), ImmutableList.of(idSlot, sumAlias), true,
+ java.util.Optional.empty(), scan);
+ LogicalProject<Plan> project = new LogicalProject<>(ImmutableList.of(idSlot, s100), agg);
+ String carrierName = IvmUtil.ivmAggHiddenColumnName(0, "SUM");
+
+ // An MV that has the carrier keeps it: the observable output and state column do not change.
+ MTMV layoutWithCarrier = Mockito.mock(MTMV.class);
+ Mockito.when(layoutWithCarrier.getColumn(carrierName)).thenReturn(new Column());
+ Plan withCarrier = new IvmNormalizeMTMV().rewriteRoot(project,
+ newJobContextForRoot(project, true, Collections.emptySet(),
+ java.util.Optional.of(IvmRewriteContext.normalize(layoutWithCarrier))));
+ Assertions.assertTrue(withCarrier.getOutput().stream()
+ .anyMatch(slot -> carrierName.equals(slot.getName())));
+
+ // An MV without it owns which columns exist, so the refresh must not add one the MV does not
+ // have; apply then reads the MV's own visible column instead.
+ MTMV layoutWithoutCarrier = Mockito.mock(MTMV.class);
+ Mockito.when(layoutWithoutCarrier.getColumn(carrierName)).thenReturn(null);
+ Plan withoutCarrier = new IvmNormalizeMTMV().rewriteRoot(project,
+ newJobContextForRoot(project, true, Collections.emptySet(),
+ java.util.Optional.of(IvmRewriteContext.normalize(layoutWithoutCarrier))));
+ Assertions.assertFalse(withoutCarrier.getOutput().stream()
+ .anyMatch(slot -> carrierName.equals(slot.getName())));
+ }
+
@Test
void testRepeatUnderAggregateDoesNotOwnRowId() {
Slot idSlot = scan.getOutput().get(0);
diff --git a/regression-test/data/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_1.out b/regression-test/data/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_1.out
new file mode 100644
index 00000000000..ace0a73ecdc
--- /dev/null
+++ b/regression-test/data/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_1.out
@@ -0,0 +1,1023 @@
+-- This file is automatically generated. You should know what you did if you want to edit this
+-- !sum_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+s100 bigint Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_SUM_COL__ bigint Yes false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !cnt_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+c1 bigint No false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !min_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+m2 bigint Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_MIN_COL__ int Yes false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !max_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+m1 bigint Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_MAX_COL__ int Yes false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !list_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+n bigint No false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COLLECT_LIST_COL__ array<int> No false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !div_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+d double Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_SUM_COL__ bigint Yes false \N NONE
+__DORIS_IVM_AGG_1_COUNT_COL__ bigint No false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !cast_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+d double Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_SUM_COL__ bigint Yes false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !scalar_desc --
+__DORIS_IVM_ROW_ID_COL__ tinyint No true \N
+s100 bigint Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_SUM_COL__ bigint Yes false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !sum_avg_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+s100 bigint Yes false \N NONE
+a200 double Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_SUM_COL__ bigint Yes false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !sum_avg_mul_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+p double Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_SUM_COL__ bigint Yes false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !floor_sum_div_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+fs decimal(28,0) Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_SUM_COL__ bigint Yes false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !plain_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+s bigint Yes false \N NONE
+s100 bigint Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !cnt_star_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+c2 bigint No false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !avg_round_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+a double Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_SUM_COL__ bigint Yes false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !agg_arg_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+s bigint Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !key_expr_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+uk varchar(65533) Yes false \N NONE
+s bigint Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !initial_sum --
+1 3000
+2 7000
+3 \N
+
+-- !initial_sum_source --
+1 3000
+2 7000
+3 \N
+
+-- !initial_cnt --
+1 3
+2 3
+3 1
+
+-- !initial_cnt_source --
+1 3
+2 3
+3 1
+
+-- !initial_min --
+1 20
+2 60
+3 \N
+
+-- !initial_min_source --
+1 20
+2 60
+3 \N
+
+-- !initial_max --
+1 21
+2 41
+3 \N
+
+-- !initial_max_source --
+1 21
+2 41
+3 \N
+
+-- !initial_list --
+1 2
+2 2
+3 0
+
+-- !initial_list_source --
+1 2
+2 2
+3 0
+
+-- !initial_div --
+1 15
+2 35
+3 \N
+
+-- !initial_div_source --
+1 15
+2 35
+3 \N
+
+-- !initial_scalar --
+10000
+
+-- !initial_scalar_source --
+10000
+
+-- !initial_sum_avg --
+1 3000 3000
+2 7000 7000
+3 \N \N
+
+-- !initial_sum_avg_source --
+1 3000 3000
+2 7000 7000
+3 \N \N
+
+-- !initial_sum_avg_mul --
+1 450
+2 2450
+3 \N
+
+-- !initial_sum_avg_mul_source --
+1 450
+2 2450
+3 \N
+
+-- !initial_floor_sum_div --
+1 81
+2 161
+3 \N
+
+-- !initial_floor_sum_div_source --
+1 81
+2 161
+3 \N
+
+-- !after_insert_sum --
+1 6000
+2 10500
+3 \N
+
+-- !after_insert_sum_source --
+1 6000
+2 10500
+3 \N
+
+-- !after_insert_cnt --
+1 4
+2 4
+3 1
+
+-- !after_insert_cnt_source --
+1 4
+2 4
+3 1
+
+-- !after_insert_min --
+1 20
+2 60
+3 \N
+
+-- !after_insert_min_source --
+1 20
+2 60
+3 \N
+
+-- !after_insert_max --
+1 31
+2 41
+3 \N
+
+-- !after_insert_max_source --
+1 31
+2 41
+3 \N
+
+-- !after_insert_list --
+1 3
+2 3
+3 0
+
+-- !after_insert_list_source --
+1 3
+2 3
+3 0
+
+-- !after_insert_div --
+1 20
+2 35
+3 \N
+
+-- !after_insert_div_source --
+1 20
+2 35
+3 \N
+
+-- !after_insert_scalar --
+16500
+
+-- !after_insert_scalar_source --
+16500
+
+-- !after_insert_sum_avg --
+1 6000 4000
+2 10500 7000
+3 \N \N
+
+-- !after_insert_sum_avg_source --
+1 6000 4000
+2 10500 7000
+3 \N \N
+
+-- !after_insert_sum_avg_mul --
+1 1200
+2 3675
+3 \N
+
+-- !after_insert_sum_avg_mul_source --
+1 1200
+2 3675
+3 \N
+
+-- !after_insert_floor_sum_div --
+1 141
+2 231
+3 \N
+
+-- !after_insert_floor_sum_div_source --
+1 141
+2 231
+3 \N
+
+-- !after_update_sum --
+1 6500
+2 10500
+3 \N
+
+-- !after_update_sum_source --
+1 6500
+2 10500
+3 \N
+
+-- !after_update_cnt --
+1 4
+2 4
+3 1
+
+-- !after_update_cnt_source --
+1 4
+2 4
+3 1
+
+-- !after_update_min --
+1 20
+2 60
+3 \N
+
+-- !after_update_min_source --
+1 20
+2 60
+3 \N
+
+-- !after_update_max --
+1 31
+2 41
+3 \N
+
+-- !after_update_max_source --
+1 31
+2 41
+3 \N
+
+-- !after_update_list --
+1 3
+2 3
+3 0
+
+-- !after_update_list_source --
+1 3
+2 3
+3 0
+
+-- !after_update_div --
+1 21.666666666666668
+2 35
+3 \N
+
+-- !after_update_div_source --
+1 21.666666666666668
+2 35
+3 \N
+
+-- !after_update_scalar --
+17000
+
+-- !after_update_scalar_source --
+17000
+
+-- !after_update_sum_avg --
+1 6500 4333.333333333334
+2 10500 7000
+3 \N \N
+
+-- !after_update_sum_avg_source --
+1 6500 4333.333333333334
+2 10500 7000
+3 \N \N
+
+-- !after_update_sum_avg_mul --
+1 1408.3333333333333
+2 3675
+3 \N
+
+-- !after_update_sum_avg_mul_source --
+1 1408.3333333333333
+2 3675
+3 \N
+
+-- !after_update_floor_sum_div --
+1 151
+2 231
+3 \N
+
+-- !after_update_floor_sum_div_source --
+1 151
+2 231
+3 \N
+
+-- !after_delete_sum --
+1 10500
+2 7000
+3 \N
+
+-- !after_delete_sum_source --
+1 10500
+2 7000
+3 \N
+
+-- !after_delete_cnt --
+1 5
+2 3
+3 1
+
+-- !after_delete_cnt_source --
+1 5
+2 3
+3 1
+
+-- !after_delete_min --
+1 20
+2 60
+3 \N
+
+-- !after_delete_min_source --
+1 20
+2 60
+3 \N
+
+-- !after_delete_max --
+1 41
+2 41
+3 \N
+
+-- !after_delete_max_source --
+1 41
+2 41
+3 \N
+
+-- !after_delete_list --
+1 4
+2 2
+3 0
+
+-- !after_delete_list_source --
+1 4
+2 2
+3 0
+
+-- !after_delete_div --
+1 26.25
+2 35
+3 \N
+
+-- !after_delete_div_source --
+1 26.25
+2 35
+3 \N
+
+-- !after_delete_scalar --
+17500
+
+-- !after_delete_scalar_source --
+17500
+
+-- !after_delete_sum_avg --
+1 10500 5250
+2 7000 7000
+3 \N \N
+
+-- !after_delete_sum_avg_source --
+1 10500 5250
+2 7000 7000
+3 \N \N
+
+-- !after_delete_sum_avg_mul --
+1 2756.25
+2 2450
+3 \N
+
+-- !after_delete_sum_avg_mul_source --
+1 2756.25
+2 2450
+3 \N
+
+-- !after_delete_floor_sum_div --
+1 231
+2 161
+3 \N
+
+-- !after_delete_floor_sum_div_source --
+1 231
+2 161
+3 \N
+
+-- !after_group_empty_sum --
+1 10500
+2 7000
+4 5000
+
+-- !after_group_empty_sum_source --
+1 10500
+2 7000
+4 5000
+
+-- !after_group_empty_cnt --
+1 5
+2 3
+4 2
+
+-- !after_group_empty_cnt_source --
+1 5
+2 3
+4 2
+
+-- !after_group_empty_min --
+1 20
+2 60
+4 100
+
+-- !after_group_empty_min_source --
+1 20
+2 60
+4 100
+
+-- !after_group_empty_max --
+1 41
+2 41
+4 51
+
+-- !after_group_empty_max_source --
+1 41
+2 41
+4 51
+
+-- !after_group_empty_list --
+1 4
+2 2
+4 1
+
+-- !after_group_empty_list_source --
+1 4
+2 2
+4 1
+
+-- !after_group_empty_div --
+1 26.25
+2 35
+4 50
+
+-- !after_group_empty_div_source --
+1 26.25
+2 35
+4 50
+
+-- !after_group_empty_scalar --
+22500
+
+-- !after_group_empty_scalar_source --
+22500
+
+-- !after_group_empty_sum_avg --
+1 10500 5250
+2 7000 7000
+4 5000 10000
+
+-- !after_group_empty_sum_avg_source --
+1 10500 5250
+2 7000 7000
+4 5000 10000
+
+-- !after_group_empty_sum_avg_mul --
+1 2756.25
+2 2450
+4 2500
+
+-- !after_group_empty_sum_avg_mul_source --
+1 2756.25
+2 2450
+4 2500
+
+-- !after_group_empty_floor_sum_div --
+1 231
+2 161
+4 121
+
+-- !after_group_empty_floor_sum_div_source --
+1 231
+2 161
+4 121
+
+-- !final_sum --
+1 10500
+2 7000
+3 7000
+4 5000
+
+-- !final_sum_source --
+1 10500
+2 7000
+3 7000
+4 5000
+
+-- !final_cnt --
+1 5
+2 3
+3 2
+4 2
+
+-- !final_cnt_source --
+1 5
+2 3
+3 2
+4 2
+
+-- !final_min --
+1 20
+2 60
+3 140
+4 100
+
+-- !final_min_source --
+1 20
+2 60
+3 140
+4 100
+
+-- !final_max --
+1 41
+2 41
+3 71
+4 51
+
+-- !final_max_source --
+1 41
+2 41
+3 71
+4 51
+
+-- !final_list --
+1 4
+2 2
+3 1
+4 1
+
+-- !final_list_source --
+1 4
+2 2
+3 1
+4 1
+
+-- !final_div --
+1 26.25
+2 35
+3 70
+4 50
+
+-- !final_div_source --
+1 26.25
+2 35
+3 70
+4 50
+
+-- !final_cast --
+1 105
+2 70
+3 70
+4 50
+
+-- !final_cast_source --
+1 105
+2 70
+3 70
+4 50
+
+-- !final_scalar --
+29500
+
+-- !final_scalar_source --
+29500
+
+-- !final_sum_avg --
+1 10500 5250
+2 7000 7000
+3 7000 14000
+4 5000 10000
+
+-- !final_sum_avg_source --
+1 10500 5250
+2 7000 7000
+3 7000 14000
+4 5000 10000
+
+-- !final_sum_avg_mul --
+1 2756.25
+2 2450
+3 4900
+4 2500
+
+-- !final_sum_avg_mul_source --
+1 2756.25
+2 2450
+3 4900
+4 2500
+
+-- !final_floor_sum_div --
+1 231
+2 161
+3 161
+4 121
+
+-- !final_floor_sum_div_source --
+1 231
+2 161
+3 161
+4 121
+
+-- !final_avg_round --
+1 26.25
+2 35
+3 70
+4 50
+
+-- !final_avg_round_source --
+1 26.25
+2 35
+3 70
+4 50
+
+-- !final_cnt_star --
+1 8
+2 4
+3 2
+4 2
+
+-- !final_cnt_star_source --
+1 8
+2 4
+3 2
+4 2
+
+-- !final_agg_arg --
+1 10500
+2 7000
+3 7000
+4 5000
+
+-- !final_agg_arg_source --
+1 10500
+2 7000
+3 7000
+4 5000
+
+-- !final_key_expr --
+1 105
+2 70
+3 70
+4 50
+
+-- !final_key_expr_source --
+1 105
+2 70
+3 70
+4 50
+
+-- !final_plain --
+1 105 10500
+2 70 7000
+3 70 7000
+4 50 5000
+
+-- !final_plain_source --
+1 105 10500
+2 70 7000
+3 70 7000
+4 50 5000
+
+-- !complete_sum --
+1 10500
+2 7000
+3 7000
+4 5000
+
+-- !complete_sum_source --
+1 10500
+2 7000
+3 7000
+4 5000
+
+-- !complete_cnt --
+1 5
+2 3
+3 2
+4 2
+
+-- !complete_cnt_source --
+1 5
+2 3
+3 2
+4 2
+
+-- !complete_min --
+1 20
+2 60
+3 140
+4 100
+
+-- !complete_min_source --
+1 20
+2 60
+3 140
+4 100
+
+-- !complete_max --
+1 41
+2 41
+3 71
+4 51
+
+-- !complete_max_source --
+1 41
+2 41
+3 71
+4 51
+
+-- !complete_list --
+1 4
+2 2
+3 1
+4 1
+
+-- !complete_list_source --
+1 4
+2 2
+3 1
+4 1
+
+-- !complete_div --
+1 26.25
+2 35
+3 70
+4 50
+
+-- !complete_div_source --
+1 26.25
+2 35
+3 70
+4 50
+
+-- !complete_cast --
+1 105
+2 70
+3 70
+4 50
+
+-- !complete_cast_source --
+1 105
+2 70
+3 70
+4 50
+
+-- !complete_scalar --
+29500
+
+-- !complete_scalar_source --
+29500
+
+-- !complete_sum_avg --
+1 10500 5250
+2 7000 7000
+3 7000 14000
+4 5000 10000
+
+-- !complete_sum_avg_source --
+1 10500 5250
+2 7000 7000
+3 7000 14000
+4 5000 10000
+
+-- !complete_sum_avg_mul --
+1 2756.25
+2 2450
+3 4900
+4 2500
+
+-- !complete_sum_avg_mul_source --
+1 2756.25
+2 2450
+3 4900
+4 2500
+
+-- !complete_floor_sum_div --
+1 231
+2 161
+3 161
+4 121
+
+-- !complete_floor_sum_div_source --
+1 231
+2 161
+3 161
+4 121
+
+-- !complete_plain --
+1 105 10500
+2 70 7000
+3 70 7000
+4 50 5000
+
+-- !complete_plain_source --
+1 105 10500
+2 70 7000
+3 70 7000
+4 50 5000
+
+-- !complete_cnt_star --
+1 8
+2 4
+3 2
+4 2
+
+-- !complete_cnt_star_source --
+1 8
+2 4
+3 2
+4 2
+
+-- !complete_avg_round --
+1 26.25
+2 35
+3 70
+4 50
+
+-- !complete_avg_round_source --
+1 26.25
+2 35
+3 70
+4 50
+
+-- !complete_agg_arg --
+1 10500
+2 7000
+3 7000
+4 5000
+
+-- !complete_agg_arg_source --
+1 10500
+2 7000
+3 7000
+4 5000
+
+-- !complete_key_expr --
+1 105
+2 70
+3 70
+4 50
+
+-- !complete_key_expr_source --
+1 105
+2 70
+3 70
+4 50
+
diff --git a/regression-test/data/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_2.out b/regression-test/data/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_2.out
new file mode 100644
index 00000000000..0c772bad147
--- /dev/null
+++ b/regression-test/data/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_2.out
@@ -0,0 +1,279 @@
+-- This file is automatically generated. You should know what you did if you want to edit this
+-- !full_keys_desc --
+k int Yes true \N
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+s100 bigint Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_SUM_COL__ bigint Yes false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !full_keys_initial --
+1 3000
+2 3000
+
+-- !full_keys_initial_source --
+1 3000
+2 3000
+
+-- !full_keys_after_insert --
+1 9000
+2 3000
+
+-- !full_keys_after_insert_source --
+1 9000
+2 3000
+
+-- !full_keys_complete --
+1 9000
+2 3000
+
+-- !full_keys_complete_source --
+1 9000
+2 3000
+
+-- !sumx_sumy_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+p bigint Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_1_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_SUM_COL__ bigint Yes false \N NONE
+__DORIS_IVM_AGG_1_SUM_COL__ bigint Yes false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !avgx_avgy_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+p double Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_SUM_COL__ bigint Yes false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_1_SUM_COL__ bigint Yes false \N NONE
+__DORIS_IVM_AGG_1_COUNT_COL__ bigint No false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !pair_initial_sumx_sumy --
+1 240
+2 490
+3 \N
+
+-- !pair_initial_sumx_sumy_source --
+1 240
+2 490
+3 \N
+
+-- !pair_initial_avgx_avgy --
+1 60
+2 245
+3 \N
+
+-- !pair_initial_avgx_avgy_source --
+1 60
+2 245
+3 \N
+
+-- !pair_after_update_sumx_sumy --
+1 315
+2 490
+3 \N
+
+-- !pair_after_update_sumx_sumy_source --
+1 315
+2 490
+3 \N
+
+-- !pair_after_update_avgx_avgy --
+1 78.75
+2 245
+3 \N
+
+-- !pair_after_update_avgx_avgy_source --
+1 78.75
+2 245
+3 \N
+
+-- !pair_after_delete_sumx_sumy --
+1 315
+2 1440
+3 \N
+
+-- !pair_after_delete_sumx_sumy_source --
+1 315
+2 1440
+3 \N
+
+-- !pair_after_delete_avgx_avgy --
+1 78.75
+2 360
+3 \N
+
+-- !pair_after_delete_avgx_avgy_source --
+1 78.75
+2 360
+3 \N
+
+-- !pair_after_resurrect_sumx_sumy --
+1 315
+2 1440
+3 390
+
+-- !pair_after_resurrect_sumx_sumy_source --
+1 315
+2 1440
+3 390
+
+-- !pair_after_resurrect_avgx_avgy --
+1 78.75
+2 360
+3 195
+
+-- !pair_after_resurrect_avgx_avgy_source --
+1 78.75
+2 360
+3 195
+
+-- !pair_complete_sumx_sumy --
+1 315
+2 1440
+3 390
+
+-- !pair_complete_sumx_sumy_source --
+1 315
+2 1440
+3 390
+
+-- !pair_complete_avgx_avgy --
+1 78.75
+2 360
+3 195
+
+-- !pair_complete_avgx_avgy_source --
+1 78.75
+2 360
+3 195
+
+-- !clamped_key_desc --
+__DORIS_IVM_ROW_ID_COL__ tinyint No true \N
+m varchar(65533) Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !clamped_key --
+aaa
+
+-- !clamped_key_source --
+aaa
+
+-- !clamped_key_after_delete --
+aaaa
+
+-- !clamped_key_after_delete_source --
+aaaa
+
+-- !alias_collision_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+sum(v) bigint Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_SUM_COL__ bigint Yes false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !alias_collision_initial --
+1 200
+
+-- !alias_collision_initial_source --
+1 200
+
+-- !alias_collision_after_insert --
+1 500
+
+-- !alias_collision_after_insert_source --
+1 500
+
+-- !alias_collision_after_delete --
+1 1000
+
+-- !alias_collision_after_delete_source --
+1 1000
+
+-- !alias_collision_complete --
+1 1000
+
+-- !alias_collision_complete_source --
+1 1000
+
+-- !cast_collision_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+sum(d) decimal(20,0) Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_SUM_COL__ decimal(38,2) Yes false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !cast_collision_initial --
+1 2
+
+-- !cast_collision_initial_source --
+1 2
+
+-- !cast_collision_after_insert --
+1 3
+
+-- !cast_collision_after_insert_source --
+1 3
+
+-- !cast_collision_complete --
+1 3
+
+-- !cast_collision_complete_source --
+1 3
+
+-- !min_boundary_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+m2 bigint Yes false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_MIN_COL__ int Yes false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !min_boundary_initial --
+1 20
+
+-- !min_boundary_initial_source --
+1 20
+
+-- !min_boundary_after_strict_failure --
+1 20
+
+-- !min_boundary_after_fallback --
+1 40
+
+-- !min_boundary_after_fallback_source --
+1 40
+
+-- !min_boundary_after_fallback_incremental --
+1 40
+
+-- !min_boundary_after_fallback_incremental_source --
+1 40
+
diff --git a/regression-test/data/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_3.out b/regression-test/data/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_3.out
new file mode 100644
index 00000000000..72fe16d8136
--- /dev/null
+++ b/regression-test/data/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_3.out
@@ -0,0 +1,201 @@
+-- This file is automatically generated. You should know what you did if you want to edit this
+-- !array_agg_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+lst array<int> No false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_ARRAY_AGG_COL__ array<int> No false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !collect_list_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+lst array<int> No false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_COLLECT_LIST_COL__ array<int> No false \N NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !bitmap_union_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+b bigint No false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_BITMAP_UNION_COL__ bitmap No false NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !bitmap_union_count_desc --
+__DORIS_IVM_ROW_ID_COL__ int Yes true \N
+k int Yes false \N NONE
+b bigint No false \N NONE
+__DORIS_IVM_AGG_COUNT_COL__ bigint No false \N NONE
+__DORIS_IVM_AGG_0_BITMAP_UNION_COL__ bitmap No false NONE
+__DORIS_DELETE_SIGN__ tinyint No false 0 NONE
+__DORIS_VERSION_COL__ bigint No false 0 NONE
+__DORIS_SEQUENCE_COL__ bigint Yes false \N NONE
+
+-- !initial_array_agg --
+1 [10, 20]
+2 [30, 40]
+3 [null]
+
+-- !initial_array_agg_source --
+1 [10, 20]
+2 [30, 40]
+3 [null]
+
+-- !initial_collect_list --
+1 [10, 20]
+2 [30, 40]
+3 []
+
+-- !initial_collect_list_source --
+1 [10, 20]
+2 [30, 40]
+3 []
+
+-- !initial_bitmap_union --
+1 2
+2 2
+3 0
+
+-- !initial_bitmap_union_source --
+1 2
+2 2
+3 0
+
+-- !initial_bitmap_union_count --
+1 2
+2 2
+3 0
+
+-- !initial_bitmap_union_count_source --
+1 2
+2 2
+3 0
+
+-- !after_insert_array_agg --
+1 [10, 20, 30]
+2 [30, 35, 40]
+3 [null]
+
+-- !after_insert_array_agg_source --
+1 [10, 20, 30]
+2 [30, 35, 40]
+3 [null]
+
+-- !after_insert_collect_list --
+1 [10, 20, 30]
+2 [30, 35, 40]
+3 []
+
+-- !after_insert_collect_list_source --
+1 [10, 20, 30]
+2 [30, 35, 40]
+3 []
+
+-- !after_insert_bitmap_union --
+1 3
+2 3
+3 0
+
+-- !after_insert_bitmap_union_source --
+1 3
+2 3
+3 0
+
+-- !after_insert_bitmap_union_count --
+1 3
+2 3
+3 0
+
+-- !after_insert_bitmap_union_count_source --
+1 3
+2 3
+3 0
+
+-- !after_update_array_agg --
+1 [10, 25, 30]
+2 [30, 35, 40]
+3 [null]
+
+-- !after_update_array_agg_source --
+1 [10, 25, 30]
+2 [30, 35, 40]
+3 [null]
+
+-- !after_update_collect_list --
+1 [10, 25, 30]
+2 [30, 35, 40]
+3 []
+
+-- !after_update_collect_list_source --
+1 [10, 25, 30]
+2 [30, 35, 40]
+3 []
+
+-- !after_delete_array_agg --
+1 [10, 25, 30, 40]
+2 [30, 40]
+3 [null]
+
+-- !after_delete_array_agg_source --
+1 [10, 25, 30, 40]
+2 [30, 40]
+3 [null]
+
+-- !after_delete_collect_list --
+1 [10, 25, 30, 40]
+2 [30, 40]
+3 []
+
+-- !after_delete_collect_list_source --
+1 [10, 25, 30, 40]
+2 [30, 40]
+3 []
+
+-- !complete_array_agg --
+1 [10, 25, 30, 40]
+2 [30, 40]
+3 [null]
+
+-- !complete_array_agg_source --
+1 [10, 25, 30, 40]
+2 [30, 40]
+3 [null]
+
+-- !complete_collect_list --
+1 [10, 25, 30, 40]
+2 [30, 40]
+3 []
+
+-- !complete_collect_list_source --
+1 [10, 25, 30, 40]
+2 [30, 40]
+3 []
+
+-- !complete_bitmap_union --
+1 4
+2 2
+3 0
+
+-- !complete_bitmap_union_source --
+1 4
+2 2
+3 0
+
+-- !complete_bitmap_union_count --
+1 4
+2 2
+3 0
+
+-- !complete_bitmap_union_count_source --
+1 4
+2 2
+3 0
+
diff --git a/regression-test/suites/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_1.groovy b/regression-test/suites/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_1.groovy
new file mode 100644
index 00000000000..353e4672d5b
--- /dev/null
+++ b/regression-test/suites/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_1.groovy
@@ -0,0 +1,668 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_ivm_agg_expr_over_agg_1") {
+ // Transparent rewrite could answer a query from one of the views under test, which would stop the
+ // base-table comparisons in this suite from being an oracle.
+ sql """set enable_materialized_view_rewrite = false"""
+
+ // =========================================================
+ // A scalar expression wrapped around an aggregate result, as
+ // in SELECT k, SUM(v) * 100 FROM t GROUP BY k, must stay
+ // incrementally maintainable.
+ //
+ // Apply merges the old MV state in the state domain and then
+ // re-applies the outer expression:
+ // new.s100 = f(apply(old_mv.sum_v, delta.sum_v))
+ // so the MV must persist a column carrying SUM(v) itself. That
+ // column is the visible aggregate output when the select list
+ // projects it (SELECT SUM(v) AS s, SUM(v) * 100) and a
+ // materialized hidden column when an upper expression consumes
+ // it without projecting it (SELECT SUM(v) * 100).
+ //
+ // These cases verify both halves of the invariant:
+ // * the hidden layout, via DESC (the dropped state column is
+ // materialized, once per aggregate state, reusing existing
+ // columns when they already carry it);
+ // * the merged values, through INSERT/UPDATE/DELETE and
+ // incremental refreshes.
+ //
+ // NOTE: set show_hidden_columns=true right before a DESC only —
+ // enabling it earlier puts the session in debug mode and blocks
+ // CREATE MATERIALIZED VIEW.
+ // =========================================================
+
+ def refreshIncremental = { mv ->
+ sql """REFRESH MATERIALIZED VIEW ${mv} INCREMENTAL"""
+ waitingMTMVTaskFinishedByMvName(mv)
+ }
+
+ sql """drop materialized view if exists test_ivm_expr_over_agg_sum;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_cnt;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_min;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_max;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_list;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_div;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_cast;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_scalar;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_sum_avg;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_sum_avg_mul;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_floor_sum_div;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_plain;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_cnt_star;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_avg_round;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_agg_arg;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_key_expr;"""
+ sql """drop table if exists test_ivm_expr_over_agg_base;"""
+
+ sql """
+ CREATE TABLE test_ivm_expr_over_agg_base (
+ id INT,
+ k INT,
+ v INT
+ )
+ UNIQUE KEY(id)
+ DISTRIBUTED BY HASH(id) BUCKETS 2
+ PROPERTIES (
+ "replication_num" = "1",
+ "binlog.enable" = "true",
+ "binlog.format" = "ROW", "binlog.need_historical_value" = "true",
+ "enable_unique_key_merge_on_write" = "true"
+ );
+ """
+
+ // =========================================================
+ // Part 1: hidden layout of the wrapped-aggregate shapes
+ // =========================================================
+
+ // SUM(v) * 100: SUM's own value is its mergeable state and the visible column is
+ // consumed by the outer expression, so the state is materialized as _0_SUM_COL__
+ // next to the hidden non-NULL count.
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_sum
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_sum_desc """DESC test_ivm_expr_over_agg_sum"""
+ sql """set show_hidden_columns=false"""
+
+ // COUNT(v) + 1: same for COUNT(expr), whose visible column is the count state.
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_cnt
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, COUNT(v) + 1 AS c1 FROM test_ivm_expr_over_agg_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_cnt_desc """DESC test_ivm_expr_over_agg_cnt"""
+ sql """set show_hidden_columns=false"""
+
+ // MIN(v) * 2 and MAX(v) + 1.
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_min
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, MIN(v) * 2 AS m2 FROM test_ivm_expr_over_agg_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_min_desc """DESC test_ivm_expr_over_agg_min"""
+ sql """set show_hidden_columns=false"""
+
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_max
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, MAX(v) + 1 AS m1 FROM test_ivm_expr_over_agg_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_max_desc """DESC test_ivm_expr_over_agg_max"""
+ sql """set show_hidden_columns=false"""
+
+ // ARRAY_SIZE(COLLECT_LIST(v)): the visible array is the whole aggregate state.
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_list
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, ARRAY_SIZE(COLLECT_LIST(v)) AS n FROM test_ivm_expr_over_agg_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_list_desc """DESC test_ivm_expr_over_agg_list"""
+ sql """set show_hidden_columns=false"""
+
+ // SUM(v) / COUNT(v): two states, both consumed by one expression. The visible COUNT
+ // column of the COUNT target is also SUM's hidden non-NULL count (column pool), so
+ // the shared count state is materialized exactly once.
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_div
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, SUM(v) / COUNT(v) AS d FROM test_ivm_expr_over_agg_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_div_desc """DESC test_ivm_expr_over_agg_div"""
+ sql """set show_hidden_columns=false"""
+
+ // CAST(SUM(v) AS DOUBLE) and a scalar (no GROUP BY) variant.
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_cast
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, CAST(SUM(v) AS DOUBLE) AS d FROM test_ivm_expr_over_agg_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_cast_desc """DESC test_ivm_expr_over_agg_cast"""
+ sql """set show_hidden_columns=false"""
+
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_scalar
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_scalar_desc """DESC test_ivm_expr_over_agg_scalar"""
+ sql """set show_hidden_columns=false"""
+
+ // SUM(v) * 100 next to AVG(v) * 200: AVG's hidden SUM state reuses the visible SUM
+ // column, so it must follow that column onto the single materialized carrier instead
+ // of adding a second SUM column.
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_sum_avg
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, SUM(v) * 100 AS s100, AVG(v) * 200 AS a200 FROM test_ivm_expr_over_agg_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_sum_avg_desc """DESC test_ivm_expr_over_agg_sum_avg"""
+ sql """set show_hidden_columns=false"""
+
+ // SUM(v) * AVG(v): both aggregate values are consumed by one expression. SUM's state is
+ // materialized, and AVG's hidden SUM state (which the column pool shares with the visible SUM
+ // column) follows it onto that single carrier, so AVG adds no second state column.
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_sum_avg_mul
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, SUM(v) * AVG(v) AS p FROM test_ivm_expr_over_agg_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_sum_avg_mul_desc """DESC test_ivm_expr_over_agg_sum_avg_mul"""
+ sql """set show_hidden_columns=false"""
+
+ // FLOOR((SUM(v) + 0.7) / 0.5) + 20: one expression tree consumes the SUM result, so the state is
+ // materialized and the whole tree is re-applied over the merged state on every refresh.
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_floor_sum_div
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, FLOOR((SUM(v) + 0.7) / 0.5) + 20 AS fs FROM test_ivm_expr_over_agg_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_floor_sum_div_desc """DESC test_ivm_expr_over_agg_floor_sum_div"""
+ sql """set show_hidden_columns=false"""
+
+ // =========================================================
+ // Part 2: shapes that already worked must keep their exact
+ // layout — no extra column is materialized when the visible
+ // column itself carries the state.
+ // =========================================================
+
+ // The visible SUM column is projected, so it carries the state itself.
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_plain
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, SUM(v) AS s, SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_plain_desc """DESC test_ivm_expr_over_agg_plain"""
+ sql """set show_hidden_columns=false"""
+
+ // COUNT(*) reads the group count, AVG and BITMAP_UNION_COUNT derive their visible value
+ // from hidden state: wrapping them needs no state materialization.
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_cnt_star
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, COUNT(*) * 2 AS c2 FROM test_ivm_expr_over_agg_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_cnt_star_desc """DESC test_ivm_expr_over_agg_cnt_star"""
+ sql """set show_hidden_columns=false"""
+
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_avg_round
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, ROUND(AVG(v), 2) AS a FROM test_ivm_expr_over_agg_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_avg_round_desc """DESC test_ivm_expr_over_agg_avg_round"""
+ sql """set show_hidden_columns=false"""
+
+
+
+
+
+ // The expression sits inside the aggregate, so the visible column is the state.
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_agg_arg
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, SUM(v * 100) AS s FROM test_ivm_expr_over_agg_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_agg_arg_desc """DESC test_ivm_expr_over_agg_agg_arg"""
+ sql """set show_hidden_columns=false"""
+
+ // The expression wraps the GROUP BY key, not an aggregate.
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_key_expr
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT UPPER(CAST(k AS VARCHAR)) AS uk, SUM(v) AS s FROM test_ivm_expr_over_agg_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_key_expr_desc """DESC test_ivm_expr_over_agg_key_expr"""
+ sql """set show_hidden_columns=false"""
+
+ // =========================================================
+ // Part 3: values. Every wrapped-aggregate MV above is
+ // incrementally maintainable, so a mixed INSERT / UPDATE /
+ // DELETE window per step must keep the MV equal to the same
+ // query over the base table.
+ // =========================================================
+
+ // k=1: 10, 20; k=2: 30, 40; k=3: a single NULL row.
+ sql """INSERT INTO test_ivm_expr_over_agg_base VALUES (1, 1, 10), (2, 1, 20), (3, 2, 30), (4, 2, 40), (5, 3, NULL);"""
+ refreshIncremental("test_ivm_expr_over_agg_sum")
+ refreshIncremental("test_ivm_expr_over_agg_cnt")
+ refreshIncremental("test_ivm_expr_over_agg_min")
+ refreshIncremental("test_ivm_expr_over_agg_max")
+ refreshIncremental("test_ivm_expr_over_agg_list")
+ refreshIncremental("test_ivm_expr_over_agg_div")
+ refreshIncremental("test_ivm_expr_over_agg_cast")
+ refreshIncremental("test_ivm_expr_over_agg_scalar")
+ refreshIncremental("test_ivm_expr_over_agg_sum_avg")
+ refreshIncremental("test_ivm_expr_over_agg_sum_avg_mul")
+ refreshIncremental("test_ivm_expr_over_agg_floor_sum_div")
+ refreshIncremental("test_ivm_expr_over_agg_plain")
+ refreshIncremental("test_ivm_expr_over_agg_cnt_star")
+ refreshIncremental("test_ivm_expr_over_agg_avg_round")
+ refreshIncremental("test_ivm_expr_over_agg_agg_arg")
+ refreshIncremental("test_ivm_expr_over_agg_key_expr")
+
+ order_qt_initial_sum """SELECT k, s100 FROM test_ivm_expr_over_agg_sum"""
+ order_qt_initial_sum_source """SELECT k, SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_initial_cnt """SELECT k, c1 FROM test_ivm_expr_over_agg_cnt"""
+ order_qt_initial_cnt_source """SELECT k, COUNT(v) + 1 AS c1 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_initial_min """SELECT k, m2 FROM test_ivm_expr_over_agg_min"""
+ order_qt_initial_min_source """SELECT k, MIN(v) * 2 AS m2 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_initial_max """SELECT k, m1 FROM test_ivm_expr_over_agg_max"""
+ order_qt_initial_max_source """SELECT k, MAX(v) + 1 AS m1 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_initial_list """SELECT k, n FROM test_ivm_expr_over_agg_list"""
+ order_qt_initial_list_source """SELECT k, ARRAY_SIZE(COLLECT_LIST(v)) AS n FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_initial_div """SELECT k, d FROM test_ivm_expr_over_agg_div"""
+ order_qt_initial_div_source """SELECT k, SUM(v) / COUNT(v) AS d FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_initial_scalar """SELECT s100 FROM test_ivm_expr_over_agg_scalar"""
+ order_qt_initial_scalar_source """SELECT SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base"""
+ order_qt_initial_sum_avg """SELECT k, s100, a200 FROM test_ivm_expr_over_agg_sum_avg"""
+ order_qt_initial_sum_avg_source """
+ SELECT k, SUM(v) * 100 AS s100, AVG(v) * 200 AS a200 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_initial_sum_avg_mul """SELECT k, p FROM test_ivm_expr_over_agg_sum_avg_mul"""
+ order_qt_initial_sum_avg_mul_source """
+ SELECT k, SUM(v) * AVG(v) AS p FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_initial_floor_sum_div """SELECT k, fs FROM test_ivm_expr_over_agg_floor_sum_div"""
+ order_qt_initial_floor_sum_div_source """
+ SELECT k, FLOOR((SUM(v) + 0.7) / 0.5) + 20 AS fs FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+
+ // Insert a new non-extremal value per group so later MIN/MAX merges stay incremental.
+ sql """INSERT INTO test_ivm_expr_over_agg_base VALUES (6, 1, 30), (7, 2, 35);"""
+ refreshIncremental("test_ivm_expr_over_agg_sum")
+ refreshIncremental("test_ivm_expr_over_agg_cnt")
+ refreshIncremental("test_ivm_expr_over_agg_min")
+ refreshIncremental("test_ivm_expr_over_agg_max")
+ refreshIncremental("test_ivm_expr_over_agg_list")
+ refreshIncremental("test_ivm_expr_over_agg_div")
+ refreshIncremental("test_ivm_expr_over_agg_scalar")
+ refreshIncremental("test_ivm_expr_over_agg_sum_avg")
+ refreshIncremental("test_ivm_expr_over_agg_sum_avg_mul")
+ refreshIncremental("test_ivm_expr_over_agg_floor_sum_div")
+
+ order_qt_after_insert_sum """SELECT k, s100 FROM test_ivm_expr_over_agg_sum"""
+ order_qt_after_insert_sum_source """SELECT k, SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_insert_cnt """SELECT k, c1 FROM test_ivm_expr_over_agg_cnt"""
+ order_qt_after_insert_cnt_source """SELECT k, COUNT(v) + 1 AS c1 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_insert_min """SELECT k, m2 FROM test_ivm_expr_over_agg_min"""
+ order_qt_after_insert_min_source """SELECT k, MIN(v) * 2 AS m2 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_insert_max """SELECT k, m1 FROM test_ivm_expr_over_agg_max"""
+ order_qt_after_insert_max_source """SELECT k, MAX(v) + 1 AS m1 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_insert_list """SELECT k, n FROM test_ivm_expr_over_agg_list"""
+ order_qt_after_insert_list_source """
+ SELECT k, ARRAY_SIZE(COLLECT_LIST(v)) AS n FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_insert_div """SELECT k, d FROM test_ivm_expr_over_agg_div"""
+ order_qt_after_insert_div_source """SELECT k, SUM(v) / COUNT(v) AS d FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_insert_scalar """SELECT s100 FROM test_ivm_expr_over_agg_scalar"""
+ order_qt_after_insert_scalar_source """SELECT SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base"""
+ order_qt_after_insert_sum_avg """SELECT k, s100, a200 FROM test_ivm_expr_over_agg_sum_avg"""
+ order_qt_after_insert_sum_avg_source """
+ SELECT k, SUM(v) * 100 AS s100, AVG(v) * 200 AS a200 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_insert_sum_avg_mul """SELECT k, p FROM test_ivm_expr_over_agg_sum_avg_mul"""
+ order_qt_after_insert_sum_avg_mul_source """
+ SELECT k, SUM(v) * AVG(v) AS p FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_insert_floor_sum_div """SELECT k, fs FROM test_ivm_expr_over_agg_floor_sum_div"""
+ order_qt_after_insert_floor_sum_div_source """
+ SELECT k, FLOOR((SUM(v) + 0.7) / 0.5) + 20 AS fs FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+ // The bitmap MV is refreshed in this insert-only window: deleting a non-NULL bitmap element is
+ // not incrementally maintainable (the delete guard requires COMPLETE), which is a pre-existing
+ // bitmap limitation covered by test_ivm_bitmap_runtime_fallback, not a wrapped-expression one.
+
+
+ // Update id=2 (k=1: 20 -> 25), a non-extremal value of k=1.
+ sql """INSERT INTO test_ivm_expr_over_agg_base VALUES (2, 1, 25);"""
+ refreshIncremental("test_ivm_expr_over_agg_sum")
+ refreshIncremental("test_ivm_expr_over_agg_cnt")
+ refreshIncremental("test_ivm_expr_over_agg_min")
+ refreshIncremental("test_ivm_expr_over_agg_max")
+ refreshIncremental("test_ivm_expr_over_agg_list")
+ refreshIncremental("test_ivm_expr_over_agg_div")
+ refreshIncremental("test_ivm_expr_over_agg_scalar")
+ refreshIncremental("test_ivm_expr_over_agg_sum_avg")
+ refreshIncremental("test_ivm_expr_over_agg_sum_avg_mul")
+ refreshIncremental("test_ivm_expr_over_agg_floor_sum_div")
+
+ order_qt_after_update_sum """SELECT k, s100 FROM test_ivm_expr_over_agg_sum"""
+ order_qt_after_update_sum_source """SELECT k, SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_update_cnt """SELECT k, c1 FROM test_ivm_expr_over_agg_cnt"""
+ order_qt_after_update_cnt_source """SELECT k, COUNT(v) + 1 AS c1 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_update_min """SELECT k, m2 FROM test_ivm_expr_over_agg_min"""
+ order_qt_after_update_min_source """SELECT k, MIN(v) * 2 AS m2 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_update_max """SELECT k, m1 FROM test_ivm_expr_over_agg_max"""
+ order_qt_after_update_max_source """SELECT k, MAX(v) + 1 AS m1 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_update_list """SELECT k, n FROM test_ivm_expr_over_agg_list"""
+ order_qt_after_update_list_source """
+ SELECT k, ARRAY_SIZE(COLLECT_LIST(v)) AS n FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_update_div """SELECT k, d FROM test_ivm_expr_over_agg_div"""
+ order_qt_after_update_div_source """SELECT k, SUM(v) / COUNT(v) AS d FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_update_scalar """SELECT s100 FROM test_ivm_expr_over_agg_scalar"""
+ order_qt_after_update_scalar_source """SELECT SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base"""
+ order_qt_after_update_sum_avg """SELECT k, s100, a200 FROM test_ivm_expr_over_agg_sum_avg"""
+ order_qt_after_update_sum_avg_source """
+ SELECT k, SUM(v) * 100 AS s100, AVG(v) * 200 AS a200 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_update_sum_avg_mul """SELECT k, p FROM test_ivm_expr_over_agg_sum_avg_mul"""
+ order_qt_after_update_sum_avg_mul_source """
+ SELECT k, SUM(v) * AVG(v) AS p FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_update_floor_sum_div """SELECT k, fs FROM test_ivm_expr_over_agg_floor_sum_div"""
+ order_qt_after_update_floor_sum_div_source """
+ SELECT k, FLOOR((SUM(v) + 0.7) / 0.5) + 20 AS fs FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+ // Delete the middle value of k=2 (id=7, v=35) together with a dirty insert, so the
+ // group keeps at least one live extreme and MIN/MAX stay incremental.
+ sql """DELETE FROM test_ivm_expr_over_agg_base WHERE id = 7;"""
+ sql """INSERT INTO test_ivm_expr_over_agg_base VALUES (8, 1, 40);"""
+ refreshIncremental("test_ivm_expr_over_agg_sum")
+ refreshIncremental("test_ivm_expr_over_agg_cnt")
+ refreshIncremental("test_ivm_expr_over_agg_min")
+ refreshIncremental("test_ivm_expr_over_agg_max")
+ refreshIncremental("test_ivm_expr_over_agg_list")
+ refreshIncremental("test_ivm_expr_over_agg_div")
+ refreshIncremental("test_ivm_expr_over_agg_scalar")
+ refreshIncremental("test_ivm_expr_over_agg_sum_avg")
+ refreshIncremental("test_ivm_expr_over_agg_sum_avg_mul")
+ refreshIncremental("test_ivm_expr_over_agg_floor_sum_div")
+
+ order_qt_after_delete_sum """SELECT k, s100 FROM test_ivm_expr_over_agg_sum"""
+ order_qt_after_delete_sum_source """SELECT k, SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_delete_cnt """SELECT k, c1 FROM test_ivm_expr_over_agg_cnt"""
+ order_qt_after_delete_cnt_source """SELECT k, COUNT(v) + 1 AS c1 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_delete_min """SELECT k, m2 FROM test_ivm_expr_over_agg_min"""
+ order_qt_after_delete_min_source """SELECT k, MIN(v) * 2 AS m2 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_delete_max """SELECT k, m1 FROM test_ivm_expr_over_agg_max"""
+ order_qt_after_delete_max_source """SELECT k, MAX(v) + 1 AS m1 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_delete_list """SELECT k, n FROM test_ivm_expr_over_agg_list"""
+ order_qt_after_delete_list_source """
+ SELECT k, ARRAY_SIZE(COLLECT_LIST(v)) AS n FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_delete_div """SELECT k, d FROM test_ivm_expr_over_agg_div"""
+ order_qt_after_delete_div_source """SELECT k, SUM(v) / COUNT(v) AS d FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_delete_scalar """SELECT s100 FROM test_ivm_expr_over_agg_scalar"""
+ order_qt_after_delete_scalar_source """SELECT SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base"""
+ order_qt_after_delete_sum_avg """SELECT k, s100, a200 FROM test_ivm_expr_over_agg_sum_avg"""
+ order_qt_after_delete_sum_avg_source """
+ SELECT k, SUM(v) * 100 AS s100, AVG(v) * 200 AS a200 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_delete_sum_avg_mul """SELECT k, p FROM test_ivm_expr_over_agg_sum_avg_mul"""
+ order_qt_after_delete_sum_avg_mul_source """
+ SELECT k, SUM(v) * AVG(v) AS p FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_delete_floor_sum_div """SELECT k, fs FROM test_ivm_expr_over_agg_floor_sum_div"""
+ order_qt_after_delete_floor_sum_div_source """
+ SELECT k, FLOOR((SUM(v) + 0.7) / 0.5) + 20 AS fs FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+ // Delete the only row of k=3 (a NULL row) so the group empties, and insert a new group
+ // k=4 in the same window: an emptied group must be removed, a new one created.
+ sql """DELETE FROM test_ivm_expr_over_agg_base WHERE id = 5;"""
+ sql """INSERT INTO test_ivm_expr_over_agg_base VALUES (9, 4, 50);"""
+ refreshIncremental("test_ivm_expr_over_agg_sum")
+ refreshIncremental("test_ivm_expr_over_agg_cnt")
+ refreshIncremental("test_ivm_expr_over_agg_min")
+ refreshIncremental("test_ivm_expr_over_agg_max")
+ refreshIncremental("test_ivm_expr_over_agg_list")
+ refreshIncremental("test_ivm_expr_over_agg_div")
+ refreshIncremental("test_ivm_expr_over_agg_scalar")
+ refreshIncremental("test_ivm_expr_over_agg_sum_avg")
+ refreshIncremental("test_ivm_expr_over_agg_sum_avg_mul")
+ refreshIncremental("test_ivm_expr_over_agg_floor_sum_div")
+
+ order_qt_after_group_empty_sum """SELECT k, s100 FROM test_ivm_expr_over_agg_sum"""
+ order_qt_after_group_empty_sum_source """SELECT k, SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_group_empty_cnt """SELECT k, c1 FROM test_ivm_expr_over_agg_cnt"""
+ order_qt_after_group_empty_cnt_source """SELECT k, COUNT(v) + 1 AS c1 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_group_empty_min """SELECT k, m2 FROM test_ivm_expr_over_agg_min"""
+ order_qt_after_group_empty_min_source """SELECT k, MIN(v) * 2 AS m2 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_group_empty_max """SELECT k, m1 FROM test_ivm_expr_over_agg_max"""
+ order_qt_after_group_empty_max_source """SELECT k, MAX(v) + 1 AS m1 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_group_empty_list """SELECT k, n FROM test_ivm_expr_over_agg_list"""
+ order_qt_after_group_empty_list_source """
+ SELECT k, ARRAY_SIZE(COLLECT_LIST(v)) AS n FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_group_empty_div """SELECT k, d FROM test_ivm_expr_over_agg_div"""
+ order_qt_after_group_empty_div_source """
+ SELECT k, SUM(v) / COUNT(v) AS d FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_group_empty_scalar """SELECT s100 FROM test_ivm_expr_over_agg_scalar"""
+ order_qt_after_group_empty_scalar_source """SELECT SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base"""
+ order_qt_after_group_empty_sum_avg """SELECT k, s100, a200 FROM test_ivm_expr_over_agg_sum_avg"""
+ order_qt_after_group_empty_sum_avg_source """
+ SELECT k, SUM(v) * 100 AS s100, AVG(v) * 200 AS a200 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_group_empty_sum_avg_mul """SELECT k, p FROM test_ivm_expr_over_agg_sum_avg_mul"""
+ order_qt_after_group_empty_sum_avg_mul_source """
+ SELECT k, SUM(v) * AVG(v) AS p FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_after_group_empty_floor_sum_div """SELECT k, fs FROM test_ivm_expr_over_agg_floor_sum_div"""
+ order_qt_after_group_empty_floor_sum_div_source """
+ SELECT k, FLOOR((SUM(v) + 0.7) / 0.5) + 20 AS fs FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+ // Resurrect the emptied group k=3.
+ sql """INSERT INTO test_ivm_expr_over_agg_base VALUES (10, 3, 70);"""
+ refreshIncremental("test_ivm_expr_over_agg_sum")
+ refreshIncremental("test_ivm_expr_over_agg_cnt")
+ refreshIncremental("test_ivm_expr_over_agg_min")
+ refreshIncremental("test_ivm_expr_over_agg_max")
+ refreshIncremental("test_ivm_expr_over_agg_list")
+ refreshIncremental("test_ivm_expr_over_agg_div")
+ refreshIncremental("test_ivm_expr_over_agg_scalar")
+ refreshIncremental("test_ivm_expr_over_agg_sum_avg")
+ refreshIncremental("test_ivm_expr_over_agg_sum_avg_mul")
+ refreshIncremental("test_ivm_expr_over_agg_floor_sum_div")
+ refreshIncremental("test_ivm_expr_over_agg_cast")
+ refreshIncremental("test_ivm_expr_over_agg_plain")
+ refreshIncremental("test_ivm_expr_over_agg_cnt_star")
+ refreshIncremental("test_ivm_expr_over_agg_avg_round")
+ refreshIncremental("test_ivm_expr_over_agg_agg_arg")
+ refreshIncremental("test_ivm_expr_over_agg_key_expr")
+
+ order_qt_final_sum """SELECT k, s100 FROM test_ivm_expr_over_agg_sum"""
+ order_qt_final_sum_source """SELECT k, SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_final_cnt """SELECT k, c1 FROM test_ivm_expr_over_agg_cnt"""
+ order_qt_final_cnt_source """SELECT k, COUNT(v) + 1 AS c1 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_final_min """SELECT k, m2 FROM test_ivm_expr_over_agg_min"""
+ order_qt_final_min_source """SELECT k, MIN(v) * 2 AS m2 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_final_max """SELECT k, m1 FROM test_ivm_expr_over_agg_max"""
+ order_qt_final_max_source """SELECT k, MAX(v) + 1 AS m1 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_final_list """SELECT k, n FROM test_ivm_expr_over_agg_list"""
+ order_qt_final_list_source """SELECT k, ARRAY_SIZE(COLLECT_LIST(v)) AS n FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_final_div """SELECT k, d FROM test_ivm_expr_over_agg_div"""
+ order_qt_final_div_source """SELECT k, SUM(v) / COUNT(v) AS d FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_final_cast """SELECT k, d FROM test_ivm_expr_over_agg_cast"""
+ order_qt_final_cast_source """SELECT k, CAST(SUM(v) AS DOUBLE) AS d FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_final_scalar """SELECT s100 FROM test_ivm_expr_over_agg_scalar"""
+ order_qt_final_scalar_source """SELECT SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base"""
+ order_qt_final_sum_avg """SELECT k, s100, a200 FROM test_ivm_expr_over_agg_sum_avg"""
+ order_qt_final_sum_avg_source """
+ SELECT k, SUM(v) * 100 AS s100, AVG(v) * 200 AS a200 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_final_sum_avg_mul """SELECT k, p FROM test_ivm_expr_over_agg_sum_avg_mul"""
+ order_qt_final_sum_avg_mul_source """
+ SELECT k, SUM(v) * AVG(v) AS p FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_final_floor_sum_div """SELECT k, fs FROM test_ivm_expr_over_agg_floor_sum_div"""
+ order_qt_final_floor_sum_div_source """
+ SELECT k, FLOOR((SUM(v) + 0.7) / 0.5) + 20 AS fs FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_final_avg_round """SELECT k, a FROM test_ivm_expr_over_agg_avg_round"""
+ order_qt_final_avg_round_source """SELECT k, ROUND(AVG(v), 2) AS a FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_final_cnt_star """SELECT k, c2 FROM test_ivm_expr_over_agg_cnt_star"""
+ order_qt_final_cnt_star_source """SELECT k, COUNT(*) * 2 AS c2 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_final_agg_arg """SELECT k, s FROM test_ivm_expr_over_agg_agg_arg"""
+ order_qt_final_agg_arg_source """SELECT k, SUM(v * 100) AS s FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_final_key_expr """SELECT uk, s FROM test_ivm_expr_over_agg_key_expr"""
+ order_qt_final_key_expr_source """
+ SELECT UPPER(CAST(k AS VARCHAR)) AS uk, SUM(v) AS s FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ order_qt_final_plain """SELECT k, s, s100 FROM test_ivm_expr_over_agg_plain"""
+ order_qt_final_plain_source """
+ SELECT k, SUM(v) AS s, SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+ // =========================================================
+ // Part 4: a complete refresh of every MV must produce the
+ // same values as the incremental path, so the layout created
+ // at CREATE time (the materialized state columns) is verified
+ // independently of the merge path.
+ // =========================================================
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_sum COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_sum")
+ order_qt_complete_sum """SELECT k, s100 FROM test_ivm_expr_over_agg_sum"""
+ order_qt_complete_sum_source """SELECT k, SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_cnt COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_cnt")
+ order_qt_complete_cnt """SELECT k, c1 FROM test_ivm_expr_over_agg_cnt"""
+ order_qt_complete_cnt_source """SELECT k, COUNT(v) + 1 AS c1 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_min COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_min")
+ order_qt_complete_min """SELECT k, m2 FROM test_ivm_expr_over_agg_min"""
+ order_qt_complete_min_source """SELECT k, MIN(v) * 2 AS m2 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_max COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_max")
+ order_qt_complete_max """SELECT k, m1 FROM test_ivm_expr_over_agg_max"""
+ order_qt_complete_max_source """SELECT k, MAX(v) + 1 AS m1 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_list COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_list")
+ order_qt_complete_list """SELECT k, n FROM test_ivm_expr_over_agg_list"""
+ order_qt_complete_list_source """
+ SELECT k, ARRAY_SIZE(COLLECT_LIST(v)) AS n FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_div COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_div")
+ order_qt_complete_div """SELECT k, d FROM test_ivm_expr_over_agg_div"""
+ order_qt_complete_div_source """SELECT k, SUM(v) / COUNT(v) AS d FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_cast COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_cast")
+ order_qt_complete_cast """SELECT k, d FROM test_ivm_expr_over_agg_cast"""
+ order_qt_complete_cast_source """
+ SELECT k, CAST(SUM(v) AS DOUBLE) AS d FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_scalar COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_scalar")
+ order_qt_complete_scalar """SELECT s100 FROM test_ivm_expr_over_agg_scalar"""
+ order_qt_complete_scalar_source """SELECT SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_sum_avg COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_sum_avg")
+ order_qt_complete_sum_avg """SELECT k, s100, a200 FROM test_ivm_expr_over_agg_sum_avg"""
+ order_qt_complete_sum_avg_source """
+ SELECT k, SUM(v) * 100 AS s100, AVG(v) * 200 AS a200 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_sum_avg_mul COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_sum_avg_mul")
+ order_qt_complete_sum_avg_mul """SELECT k, p FROM test_ivm_expr_over_agg_sum_avg_mul"""
+ order_qt_complete_sum_avg_mul_source """
+ SELECT k, SUM(v) * AVG(v) AS p FROM test_ivm_expr_over_agg_base GROUP BY k"""
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_floor_sum_div COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_floor_sum_div")
+ order_qt_complete_floor_sum_div """SELECT k, fs FROM test_ivm_expr_over_agg_floor_sum_div"""
+ order_qt_complete_floor_sum_div_source """
+ SELECT k, FLOOR((SUM(v) + 0.7) / 0.5) + 20 AS fs FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_plain COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_plain")
+ order_qt_complete_plain """SELECT k, s, s100 FROM test_ivm_expr_over_agg_plain"""
+ order_qt_complete_plain_source """
+ SELECT k, SUM(v) AS s, SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_cnt_star COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_cnt_star")
+ order_qt_complete_cnt_star """SELECT k, c2 FROM test_ivm_expr_over_agg_cnt_star"""
+ order_qt_complete_cnt_star_source """
+ SELECT k, COUNT(*) * 2 AS c2 FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_avg_round COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_avg_round")
+ order_qt_complete_avg_round """SELECT k, a FROM test_ivm_expr_over_agg_avg_round"""
+ order_qt_complete_avg_round_source """
+ SELECT k, ROUND(AVG(v), 2) AS a FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+
+
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_agg_arg COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_agg_arg")
+ order_qt_complete_agg_arg """SELECT k, s FROM test_ivm_expr_over_agg_agg_arg"""
+ order_qt_complete_agg_arg_source """SELECT k, SUM(v * 100) AS s FROM test_ivm_expr_over_agg_base GROUP BY k"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_key_expr COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_key_expr")
+ order_qt_complete_key_expr """SELECT uk, s FROM test_ivm_expr_over_agg_key_expr"""
+ order_qt_complete_key_expr_source """
+ SELECT UPPER(CAST(k AS VARCHAR)) AS uk, SUM(v) AS s FROM test_ivm_expr_over_agg_base GROUP BY k"""
+}
diff --git a/regression-test/suites/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_2.groovy b/regression-test/suites/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_2.groovy
new file mode 100644
index 00000000000..1b7a569dbc0
--- /dev/null
+++ b/regression-test/suites/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_2.groovy
@@ -0,0 +1,483 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+import org.awaitility.Awaitility
+import static java.util.concurrent.TimeUnit.SECONDS
+
+suite("test_ivm_agg_expr_over_agg_2") {
+ // Transparent rewrite could answer a query from one of the views under test, which would stop the
+ // base-table comparisons in this suite from being an oracle.
+ sql """set enable_materialized_view_rewrite = false"""
+
+ // =========================================================
+ // Refresh shapes around the wrapped-aggregate state carrier
+ // that need their own setup: the same shape under
+ // ivm_use_full_keys, two aggregate values consumed by one
+ // expression, an aggregate whose result is clamped into the
+ // MV key column, and a view that aliases its own expression
+ // to the aggregate's generated column name.
+ // =========================================================
+
+ // =========================================================
+ // Part 5: the same wrapped shape under ivm_use_full_keys,
+ // where normalize also materializes hidden identity-key
+ // columns into the same projection as the state carrier.
+ // Created after the windows above so its inserts stay a
+ // clean incremental window.
+ // =========================================================
+
+ def refreshIncremental = { mv ->
+ sql """REFRESH MATERIALIZED VIEW ${mv} INCREMENTAL"""
+ waitingMTMVTaskFinishedByMvName(mv)
+ }
+
+ sql """drop materialized view if exists test_ivm_expr_over_agg_full_keys;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_alias_collision;"""
+ sql """drop table if exists test_ivm_expr_over_agg_alias_base;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_min_boundary;"""
+ sql """drop table if exists test_ivm_expr_over_agg_min_base;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_cast_collision;"""
+ sql """drop table if exists test_ivm_expr_over_agg_cast_base;"""
+ sql """drop table if exists test_ivm_expr_over_agg_2_base;"""
+
+ sql """
+ CREATE TABLE test_ivm_expr_over_agg_2_base (
+ id INT,
+ k INT,
+ v INT
+ )
+ UNIQUE KEY(id)
+ DISTRIBUTED BY HASH(id) BUCKETS 2
+ PROPERTIES (
+ "replication_num" = "1",
+ "binlog.enable" = "true",
+ "binlog.format" = "ROW", "binlog.need_historical_value" = "true",
+ "enable_unique_key_merge_on_write" = "true"
+ );
+ """
+ sql """INSERT INTO test_ivm_expr_over_agg_2_base VALUES (1, 1, 10), (2, 1, 20), (3, 2, 30);"""
+
+ sql """CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_full_keys
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1', 'ivm_use_full_keys' = 'true')
+ AS SELECT k, SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_2_base GROUP BY k;"""
+ sql """set show_hidden_columns=true"""
+ qt_full_keys_desc """DESC test_ivm_expr_over_agg_full_keys"""
+ sql """set show_hidden_columns=false"""
+
+ refreshIncremental("test_ivm_expr_over_agg_full_keys")
+ order_qt_full_keys_initial """SELECT k, s100 FROM test_ivm_expr_over_agg_full_keys"""
+ order_qt_full_keys_initial_source """
+ SELECT k, SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_2_base GROUP BY k"""
+
+ sql """INSERT INTO test_ivm_expr_over_agg_2_base VALUES (11, 1, 60);"""
+ refreshIncremental("test_ivm_expr_over_agg_full_keys")
+ order_qt_full_keys_after_insert """SELECT k, s100 FROM test_ivm_expr_over_agg_full_keys"""
+ order_qt_full_keys_after_insert_source """
+ SELECT k, SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_2_base GROUP BY k"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_full_keys COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_full_keys")
+ order_qt_full_keys_complete """SELECT k, s100 FROM test_ivm_expr_over_agg_full_keys"""
+ order_qt_full_keys_complete_source """
+ SELECT k, SUM(v) * 100 AS s100 FROM test_ivm_expr_over_agg_2_base GROUP BY k"""
+
+ // =========================================================
+ // Part 6: two aggregate values consumed by one expression.
+ // SUM(x) * SUM(y) needs one carrier per SUM state, while
+ // AVG(x) * AVG(y) needs none: AVG derives its visible value
+ // from hidden SUM/COUNT states, which are always persisted.
+ // =========================================================
+
+ sql """drop materialized view if exists test_ivm_expr_over_agg_sumx_sumy;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_avgx_avgy;"""
+ sql """drop table if exists test_ivm_expr_over_agg_pair_base;"""
+
+ sql """
+ CREATE TABLE test_ivm_expr_over_agg_pair_base (
+ id INT,
+ k INT,
+ x INT,
+ y INT
+ )
+ UNIQUE KEY(id)
+ DISTRIBUTED BY HASH(id) BUCKETS 2
+ PROPERTIES (
+ "replication_num" = "1",
+ "binlog.enable" = "true",
+ "binlog.format" = "ROW", "binlog.need_historical_value" = "true",
+ "enable_unique_key_merge_on_write" = "true"
+ );
+ """
+ sql """INSERT INTO test_ivm_expr_over_agg_pair_base VALUES
+ (1, 1, 10, 3), (2, 1, 20, 5), (3, 2, 30, 7), (4, 2, 40, NULL), (5, 3, NULL, 9);"""
+
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_sumx_sumy
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, SUM(x) * SUM(y) AS p FROM test_ivm_expr_over_agg_pair_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_sumx_sumy_desc """DESC test_ivm_expr_over_agg_sumx_sumy"""
+ sql """set show_hidden_columns=false"""
+
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_avgx_avgy
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, AVG(x) * AVG(y) AS p FROM test_ivm_expr_over_agg_pair_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_avgx_avgy_desc """DESC test_ivm_expr_over_agg_avgx_avgy"""
+ sql """set show_hidden_columns=false"""
+
+ refreshIncremental("test_ivm_expr_over_agg_sumx_sumy")
+ refreshIncremental("test_ivm_expr_over_agg_avgx_avgy")
+ order_qt_pair_initial_sumx_sumy """SELECT k, p FROM test_ivm_expr_over_agg_sumx_sumy"""
+ order_qt_pair_initial_sumx_sumy_source """
+ SELECT k, SUM(x) * SUM(y) AS p FROM test_ivm_expr_over_agg_pair_base GROUP BY k"""
+ order_qt_pair_initial_avgx_avgy """SELECT k, p FROM test_ivm_expr_over_agg_avgx_avgy"""
+ order_qt_pair_initial_avgx_avgy_source """
+ SELECT k, AVG(x) * AVG(y) AS p FROM test_ivm_expr_over_agg_pair_base GROUP BY k"""
+
+ // Update id=2 into (25, 6): both aggregate inputs of group k=1 change in one window.
+ sql """INSERT INTO test_ivm_expr_over_agg_pair_base VALUES (2, 1, 25, 6);"""
+ refreshIncremental("test_ivm_expr_over_agg_sumx_sumy")
+ refreshIncremental("test_ivm_expr_over_agg_avgx_avgy")
+ order_qt_pair_after_update_sumx_sumy """SELECT k, p FROM test_ivm_expr_over_agg_sumx_sumy"""
+ order_qt_pair_after_update_sumx_sumy_source """
+ SELECT k, SUM(x) * SUM(y) AS p FROM test_ivm_expr_over_agg_pair_base GROUP BY k"""
+ order_qt_pair_after_update_avgx_avgy """SELECT k, p FROM test_ivm_expr_over_agg_avgx_avgy"""
+ order_qt_pair_after_update_avgx_avgy_source """
+ SELECT k, AVG(x) * AVG(y) AS p FROM test_ivm_expr_over_agg_pair_base GROUP BY k"""
+
+ // Delete the row whose y is NULL (id=4) together with a dirty insert into k=2.
+ sql """DELETE FROM test_ivm_expr_over_agg_pair_base WHERE id = 4;"""
+ sql """INSERT INTO test_ivm_expr_over_agg_pair_base VALUES (6, 2, 50, 11);"""
+ refreshIncremental("test_ivm_expr_over_agg_sumx_sumy")
+ refreshIncremental("test_ivm_expr_over_agg_avgx_avgy")
+ order_qt_pair_after_delete_sumx_sumy """SELECT k, p FROM test_ivm_expr_over_agg_sumx_sumy"""
+ order_qt_pair_after_delete_sumx_sumy_source """
+ SELECT k, SUM(x) * SUM(y) AS p FROM test_ivm_expr_over_agg_pair_base GROUP BY k"""
+ order_qt_pair_after_delete_avgx_avgy """SELECT k, p FROM test_ivm_expr_over_agg_avgx_avgy"""
+ order_qt_pair_after_delete_avgx_avgy_source """
+ SELECT k, AVG(x) * AVG(y) AS p FROM test_ivm_expr_over_agg_pair_base GROUP BY k"""
+
+ // Group k=3 held a single row with x NULL; give it a second row with both inputs set.
+ sql """INSERT INTO test_ivm_expr_over_agg_pair_base VALUES (7, 3, 30, 4);"""
+ refreshIncremental("test_ivm_expr_over_agg_sumx_sumy")
+ refreshIncremental("test_ivm_expr_over_agg_avgx_avgy")
+ order_qt_pair_after_resurrect_sumx_sumy """SELECT k, p FROM test_ivm_expr_over_agg_sumx_sumy"""
+ order_qt_pair_after_resurrect_sumx_sumy_source """
+ SELECT k, SUM(x) * SUM(y) AS p FROM test_ivm_expr_over_agg_pair_base GROUP BY k"""
+ order_qt_pair_after_resurrect_avgx_avgy """SELECT k, p FROM test_ivm_expr_over_agg_avgx_avgy"""
+ order_qt_pair_after_resurrect_avgx_avgy_source """
+ SELECT k, AVG(x) * AVG(y) AS p FROM test_ivm_expr_over_agg_pair_base GROUP BY k"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_sumx_sumy COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_sumx_sumy")
+ order_qt_pair_complete_sumx_sumy """SELECT k, p FROM test_ivm_expr_over_agg_sumx_sumy"""
+ order_qt_pair_complete_sumx_sumy_source """
+ SELECT k, SUM(x) * SUM(y) AS p FROM test_ivm_expr_over_agg_pair_base GROUP BY k"""
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_avgx_avgy COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_avgx_avgy")
+ order_qt_pair_complete_avgx_avgy """SELECT k, p FROM test_ivm_expr_over_agg_avgx_avgy"""
+ order_qt_pair_complete_avgx_avgy_source """
+ SELECT k, AVG(x) * AVG(y) AS p FROM test_ivm_expr_over_agg_pair_base GROUP BY k"""
+
+ // =========================================================
+ // Part 8: an aggregate whose result is the MV's key column
+ // and is therefore clamped to VARCHAR(65533). The refresh
+ // plan wraps the aggregate output in binder projects that
+ // rename it and coerce it back into the MV column, so the
+ // state column must keep its own name instead of acquiring a
+ // new carrier that the MV does not have.
+ // =========================================================
+
+ sql """drop materialized view if exists test_ivm_expr_over_agg_clamped_key;"""
+ sql """drop table if exists test_ivm_expr_over_agg_str_base;"""
+
+ sql """
+ CREATE TABLE test_ivm_expr_over_agg_str_base (
+ id INT,
+ s STRING
+ )
+ UNIQUE KEY(id)
+ DISTRIBUTED BY HASH(id) BUCKETS 2
+ PROPERTIES (
+ "replication_num" = "1",
+ "binlog.enable" = "true",
+ "binlog.format" = "ROW", "binlog.need_historical_value" = "true",
+ "enable_unique_key_merge_on_write" = "true"
+ );
+ """
+ sql """INSERT INTO test_ivm_expr_over_agg_str_base VALUES (1, 'aaa'), (2, 'bbb'), (3, 'ccc');"""
+
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_clamped_key
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT MIN(s) AS m FROM test_ivm_expr_over_agg_str_base;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_clamped_key_desc """DESC test_ivm_expr_over_agg_clamped_key"""
+ sql """set show_hidden_columns=false"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_clamped_key COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_clamped_key")
+ order_qt_clamped_key """SELECT m FROM test_ivm_expr_over_agg_clamped_key"""
+ order_qt_clamped_key_source """SELECT MIN(s) AS m FROM test_ivm_expr_over_agg_str_base"""
+
+ // Delete the current minimum so the complete refresh has to rebuild the state.
+ sql """DELETE FROM test_ivm_expr_over_agg_str_base WHERE id = 1;"""
+ sql """INSERT INTO test_ivm_expr_over_agg_str_base VALUES (4, 'aaaa');"""
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_clamped_key COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_clamped_key")
+ order_qt_clamped_key_after_delete """SELECT m FROM test_ivm_expr_over_agg_clamped_key"""
+ order_qt_clamped_key_after_delete_source """SELECT MIN(s) AS m FROM test_ivm_expr_over_agg_str_base"""
+
+ // =========================================================
+ // Part 9: a view that aliases its own expression to the
+ // aggregate's generated column name. The MV column with that
+ // name holds a derived value, so the raw state still needs a
+ // materialized carrier; a same-named output is not enough.
+ // =========================================================
+
+ sql """drop materialized view if exists test_ivm_expr_over_agg_alias_collision;"""
+ sql """drop table if exists test_ivm_expr_over_agg_alias_base;"""
+
+ sql """
+ CREATE TABLE test_ivm_expr_over_agg_alias_base (
+ id INT,
+ k INT,
+ v INT
+ )
+ UNIQUE KEY(id)
+ DISTRIBUTED BY HASH(id) BUCKETS 2
+ PROPERTIES (
+ "replication_num" = "1",
+ "binlog.enable" = "true",
+ "binlog.format" = "ROW", "binlog.need_historical_value" = "true",
+ "enable_unique_key_merge_on_write" = "true"
+ );
+ """
+ sql """INSERT INTO test_ivm_expr_over_agg_alias_base VALUES (1, 1, 2);"""
+
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_alias_collision
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, SUM(v) * 100 AS `sum(v)` FROM test_ivm_expr_over_agg_alias_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_alias_collision_desc """DESC test_ivm_expr_over_agg_alias_collision"""
+ sql """set show_hidden_columns=false"""
+
+ refreshIncremental("test_ivm_expr_over_agg_alias_collision")
+ order_qt_alias_collision_initial """SELECT k, `sum(v)` FROM test_ivm_expr_over_agg_alias_collision"""
+ order_qt_alias_collision_initial_source """
+ SELECT k, SUM(v) * 100 AS `sum(v)` FROM test_ivm_expr_over_agg_alias_base GROUP BY k"""
+
+ // The alias holds 200 for SUM(v)=2; merging through it as if it were the raw state would persist 20300.
+ sql """INSERT INTO test_ivm_expr_over_agg_alias_base VALUES (2, 1, 3);"""
+ refreshIncremental("test_ivm_expr_over_agg_alias_collision")
+ order_qt_alias_collision_after_insert """SELECT k, `sum(v)` FROM test_ivm_expr_over_agg_alias_collision"""
+ order_qt_alias_collision_after_insert_source """
+ SELECT k, SUM(v) * 100 AS `sum(v)` FROM test_ivm_expr_over_agg_alias_base GROUP BY k"""
+
+ sql """DELETE FROM test_ivm_expr_over_agg_alias_base WHERE id = 1;"""
+ sql """INSERT INTO test_ivm_expr_over_agg_alias_base VALUES (3, 1, 7);"""
+ refreshIncremental("test_ivm_expr_over_agg_alias_collision")
+ order_qt_alias_collision_after_delete """SELECT k, `sum(v)` FROM test_ivm_expr_over_agg_alias_collision"""
+ order_qt_alias_collision_after_delete_source """
+ SELECT k, SUM(v) * 100 AS `sum(v)` FROM test_ivm_expr_over_agg_alias_base GROUP BY k"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_alias_collision COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_alias_collision")
+ order_qt_alias_collision_complete """SELECT k, `sum(v)` FROM test_ivm_expr_over_agg_alias_collision"""
+ order_qt_alias_collision_complete_source """
+ SELECT k, SUM(v) * 100 AS `sum(v)` FROM test_ivm_expr_over_agg_alias_base GROUP BY k"""
+
+ // =========================================================
+ // Part 10: a lossy CAST whose output takes the aggregate's
+ // generated column name. The cast is value-preserving for the
+ // type checker but not for the value, so the column can never
+ // stand in for the raw state.
+ // =========================================================
+
+ sql """drop materialized view if exists test_ivm_expr_over_agg_cast_collision;"""
+ sql """drop table if exists test_ivm_expr_over_agg_cast_base;"""
+
+ sql """
+ CREATE TABLE test_ivm_expr_over_agg_cast_base (
+ id INT,
+ k INT,
+ d DECIMAL(10, 2)
+ )
+ UNIQUE KEY(id)
+ DISTRIBUTED BY HASH(id) BUCKETS 2
+ PROPERTIES (
+ "replication_num" = "1",
+ "binlog.enable" = "true",
+ "binlog.format" = "ROW", "binlog.need_historical_value" = "true",
+ "enable_unique_key_merge_on_write" = "true"
+ );
+ """
+ sql """INSERT INTO test_ivm_expr_over_agg_cast_base VALUES (1, 1, 1.55);"""
+
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_cast_collision
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, CAST(SUM(d) AS DECIMAL(20, 0)) AS `sum(d)` FROM test_ivm_expr_over_agg_cast_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_cast_collision_desc """DESC test_ivm_expr_over_agg_cast_collision"""
+ sql """set show_hidden_columns=false"""
+
+ refreshIncremental("test_ivm_expr_over_agg_cast_collision")
+ order_qt_cast_collision_initial """SELECT k, `sum(d)` FROM test_ivm_expr_over_agg_cast_collision"""
+ order_qt_cast_collision_initial_source """
+ SELECT k, CAST(SUM(d) AS DECIMAL(20, 0)) AS `sum(d)` FROM test_ivm_expr_over_agg_cast_base GROUP BY k"""
+
+ // Merging from the rounded alias (2) instead of the raw state (1.55) persisted 157 here.
+ sql """INSERT INTO test_ivm_expr_over_agg_cast_base VALUES (2, 1, 1.55);"""
+ refreshIncremental("test_ivm_expr_over_agg_cast_collision")
+ order_qt_cast_collision_after_insert """SELECT k, `sum(d)` FROM test_ivm_expr_over_agg_cast_collision"""
+ order_qt_cast_collision_after_insert_source """
+ SELECT k, CAST(SUM(d) AS DECIMAL(20, 0)) AS `sum(d)` FROM test_ivm_expr_over_agg_cast_base GROUP BY k"""
+
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_cast_collision COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_cast_collision")
+ order_qt_cast_collision_complete """SELECT k, `sum(d)` FROM test_ivm_expr_over_agg_cast_collision"""
+ order_qt_cast_collision_complete_source """
+ SELECT k, CAST(SUM(d) AS DECIMAL(20, 0)) AS `sum(d)` FROM test_ivm_expr_over_agg_cast_base GROUP BY k"""
+
+ // =========================================================
+ // Part 11: deleting an extremal value cannot be merged into a
+ // MIN state through a carrier either, so the runtime guard
+ // must still classify it and degrade to a COMPLETE refresh
+ // instead of writing a value derived from the stale extreme.
+ // =========================================================
+
+ sql """drop materialized view if exists test_ivm_expr_over_agg_min_boundary;"""
+ sql """drop table if exists test_ivm_expr_over_agg_min_base;"""
+
+ sql """
+ CREATE TABLE test_ivm_expr_over_agg_min_base (
+ id INT,
+ k INT,
+ v INT
+ )
+ UNIQUE KEY(id)
+ DISTRIBUTED BY HASH(id) BUCKETS 2
+ PROPERTIES (
+ "replication_num" = "1",
+ "binlog.enable" = "true",
+ "binlog.format" = "ROW", "binlog.need_historical_value" = "true",
+ "enable_unique_key_merge_on_write" = "true"
+ );
+ """
+ sql """INSERT INTO test_ivm_expr_over_agg_min_base VALUES (1, 1, 10), (2, 1, 20), (3, 1, 30);"""
+
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_min_boundary
+ BUILD DEFERRED REFRESH INCREMENTAL FALLBACK ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, MIN(v) * 2 AS m2 FROM test_ivm_expr_over_agg_min_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_min_boundary_desc """DESC test_ivm_expr_over_agg_min_boundary"""
+ sql """set show_hidden_columns=false"""
+
+ // CreateTime only has second granularity and TaskId carries a random component, so selecting the
+ // newest task by ordering can return the previous refresh when both start within one second. Capture
+ // the task ids that exist before submitting a refresh and wait for the one it adds instead.
+ def taskIdsOf = {
+ sql_return_maparray("""
+ SELECT TaskId FROM tasks('type'='mv')
+ WHERE MvDatabaseName = '${context.dbName}'
+ AND MvName = 'test_ivm_expr_over_agg_min_boundary'
+ """).collect { it.TaskId.toString() } as Set
+ }
+ def submittedTask = { Set<String> before ->
+ String taskId = null
+ Awaitility.await().atMost(300, SECONDS).pollInterval(1, SECONDS).until({
+ taskId = taskIdsOf().find { !before.contains(it) }
+ return taskId != null
+ })
+ def taskResult
+ Awaitility.await().atMost(300, SECONDS).pollInterval(2, SECONDS).until({
+ taskResult = sql_return_maparray("""
+ SELECT Status, RefreshMode, IvmFallbackReason, ErrorMsg
+ FROM tasks('type'='mv')
+ WHERE MvDatabaseName = '${context.dbName}'
+ AND MvName = 'test_ivm_expr_over_agg_min_boundary'
+ AND TaskId = '${taskId}'
+ """)
+ return !taskResult.isEmpty()
+ && taskResult[0].Status.toString() != 'PENDING'
+ && taskResult[0].Status.toString() != 'RUNNING'
+ })
+ return taskResult[0]
+ }
+
+ refreshIncremental("test_ivm_expr_over_agg_min_boundary")
+ order_qt_min_boundary_initial """SELECT k, m2 FROM test_ivm_expr_over_agg_min_boundary"""
+ order_qt_min_boundary_initial_source """SELECT k, MIN(v) * 2 AS m2 FROM test_ivm_expr_over_agg_min_base GROUP BY k"""
+
+ // Delete the row that holds the current minimum: the new extreme cannot be derived from the
+ // stored state, so a strict incremental refresh must fail and name the reason.
+ sql """DELETE FROM test_ivm_expr_over_agg_min_base WHERE id = 1;"""
+ Thread.sleep(1000)
+ def beforeStrict = taskIdsOf()
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_min_boundary INCREMENTAL"""
+ def strictTask = submittedTask(beforeStrict)
+ assertEquals("FAILED", strictTask.Status.toString())
+ assertEquals("MIN_MAX_BOUNDARY_HIT", strictTask.IvmFallbackReason.toString())
+ order_qt_min_boundary_after_strict_failure """SELECT k, m2 FROM test_ivm_expr_over_agg_min_boundary"""
+
+ // With the fallback allowed, the refresh recomputes completely and matches the source query.
+ def beforeFallback = taskIdsOf()
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_min_boundary INCREMENTAL FALLBACK"""
+ def fallbackTask = submittedTask(beforeFallback)
+ assertEquals("SUCCESS", fallbackTask.Status.toString())
+ assertEquals("COMPLETE", fallbackTask.RefreshMode.toString())
+ assertEquals("MIN_MAX_BOUNDARY_HIT", fallbackTask.IvmFallbackReason.toString())
+ order_qt_min_boundary_after_fallback """SELECT k, m2 FROM test_ivm_expr_over_agg_min_boundary"""
+ order_qt_min_boundary_after_fallback_source """SELECT k, MIN(v) * 2 AS m2 FROM test_ivm_expr_over_agg_min_base GROUP BY k"""
+
+ // The fallback recomputed the whole MV, so its carrier must hold the rebuilt minimum: insert a
+ // non-boundary value and merge it incrementally. Reading a carrier left at the deleted extreme (10)
+ // would report 20 here instead of the expected 40.
+ sql """INSERT INTO test_ivm_expr_over_agg_min_base VALUES (4, 1, 25);"""
+ def beforePostFallback = taskIdsOf()
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_min_boundary INCREMENTAL"""
+ def postFallbackTask = submittedTask(beforePostFallback)
+ assertEquals("SUCCESS", postFallbackTask.Status.toString())
+ order_qt_min_boundary_after_fallback_incremental """SELECT k, m2 FROM test_ivm_expr_over_agg_min_boundary"""
+ order_qt_min_boundary_after_fallback_incremental_source """
+ SELECT k, MIN(v) * 2 AS m2 FROM test_ivm_expr_over_agg_min_base GROUP BY k"""
+}
diff --git a/regression-test/suites/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_3.groovy b/regression-test/suites/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_3.groovy
new file mode 100644
index 00000000000..8a4be5c1c97
--- /dev/null
+++ b/regression-test/suites/mtmv_p0/ivm/test_ivm_agg_expr_over_agg_3.groovy
@@ -0,0 +1,187 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_ivm_agg_expr_over_agg_3") {
+ // Transparent rewrite could answer a query from one of the views under test, which would stop the
+ // base-table comparisons in this suite from being an oracle.
+ sql """set enable_materialized_view_rewrite = false"""
+
+ // =========================================================
+ // Array and bitmap aggregate state under a wrapping
+ // expression. Each of these keeps its whole aggregate state
+ // in the visible column, which the outer expression drops, so
+ // the state is carried by a materialized column; the array
+ // cases also compare element contents rather than counts, and
+ // ARRAY_AGG keeps NULL elements where COLLECT_LIST skips them.
+ // =========================================================
+
+ def refreshIncremental = { mv ->
+ sql """REFRESH MATERIALIZED VIEW ${mv} INCREMENTAL"""
+ waitingMTMVTaskFinishedByMvName(mv)
+ }
+
+ sql """drop materialized view if exists test_ivm_expr_over_agg_3_array_agg;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_3_collect_list;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_3_bitmap_union;"""
+ sql """drop materialized view if exists test_ivm_expr_over_agg_3_bitmap_union_count;"""
+ sql """drop table if exists test_ivm_expr_over_agg_3_base;"""
+
+ sql """
+ CREATE TABLE test_ivm_expr_over_agg_3_base (
+ id INT,
+ k INT,
+ v INT
+ )
+ UNIQUE KEY(id)
+ DISTRIBUTED BY HASH(id) BUCKETS 2
+ PROPERTIES (
+ "replication_num" = "1",
+ "binlog.enable" = "true",
+ "binlog.format" = "ROW", "binlog.need_historical_value" = "true",
+ "enable_unique_key_merge_on_write" = "true"
+ );
+ """
+ sql """INSERT INTO test_ivm_expr_over_agg_3_base VALUES
+ (1, 1, 10), (2, 1, 20), (3, 2, 30), (4, 2, 40), (5, 3, NULL);"""
+
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_3_array_agg
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, ARRAY_SORT(ARRAY_AGG(v)) AS lst FROM test_ivm_expr_over_agg_3_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_array_agg_desc """DESC test_ivm_expr_over_agg_3_array_agg"""
+ sql """set show_hidden_columns=false"""
+
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_3_collect_list
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, ARRAY_SORT(COLLECT_LIST(v)) AS lst FROM test_ivm_expr_over_agg_3_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_collect_list_desc """DESC test_ivm_expr_over_agg_3_collect_list"""
+ sql """set show_hidden_columns=false"""
+
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_3_bitmap_union
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, BITMAP_COUNT(BITMAP_UNION(TO_BITMAP(v))) AS b FROM test_ivm_expr_over_agg_3_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_bitmap_union_desc """DESC test_ivm_expr_over_agg_3_bitmap_union"""
+ sql """set show_hidden_columns=false"""
+
+ sql """
+ CREATE MATERIALIZED VIEW test_ivm_expr_over_agg_3_bitmap_union_count
+ BUILD DEFERRED REFRESH INCREMENTAL ON MANUAL
+ DISTRIBUTED BY RANDOM BUCKETS 2
+ PROPERTIES ('replication_num' = '1')
+ AS SELECT k, BITMAP_UNION_COUNT(TO_BITMAP(v)) + 0 AS b FROM test_ivm_expr_over_agg_3_base GROUP BY k;
+ """
+ sql """set show_hidden_columns=true"""
+ qt_bitmap_union_count_desc """DESC test_ivm_expr_over_agg_3_bitmap_union_count"""
+ sql """set show_hidden_columns=false"""
+
+ // Initial window: all four views, the bitmaps included.
+ refreshIncremental("test_ivm_expr_over_agg_3_array_agg")
+ refreshIncremental("test_ivm_expr_over_agg_3_collect_list")
+ refreshIncremental("test_ivm_expr_over_agg_3_bitmap_union")
+ refreshIncremental("test_ivm_expr_over_agg_3_bitmap_union_count")
+ order_qt_initial_array_agg """SELECT k, lst FROM test_ivm_expr_over_agg_3_array_agg"""
+ order_qt_initial_array_agg_source """
+ SELECT k, ARRAY_SORT(ARRAY_AGG(v)) AS lst FROM test_ivm_expr_over_agg_3_base GROUP BY k"""
+ order_qt_initial_collect_list """SELECT k, lst FROM test_ivm_expr_over_agg_3_collect_list"""
+ order_qt_initial_collect_list_source """
+ SELECT k, ARRAY_SORT(COLLECT_LIST(v)) AS lst FROM test_ivm_expr_over_agg_3_base GROUP BY k"""
+ order_qt_initial_bitmap_union """SELECT k, b FROM test_ivm_expr_over_agg_3_bitmap_union"""
+ order_qt_initial_bitmap_union_source """
+ SELECT k, BITMAP_COUNT(BITMAP_UNION(TO_BITMAP(v))) AS b FROM test_ivm_expr_over_agg_3_base GROUP BY k"""
+ order_qt_initial_bitmap_union_count """SELECT k, b FROM test_ivm_expr_over_agg_3_bitmap_union_count"""
+ order_qt_initial_bitmap_union_count_source """
+ SELECT k, BITMAP_UNION_COUNT(TO_BITMAP(v)) + 0 AS b FROM test_ivm_expr_over_agg_3_base GROUP BY k"""
+
+ // Insert-only window: a bitmap element cannot be removed incrementally, so the bitmaps are refreshed
+ // here (deleting a non-NULL element is the separate limitation covered by test_ivm_bitmap_runtime_fallback).
+ sql """INSERT INTO test_ivm_expr_over_agg_3_base VALUES (6, 1, 30), (7, 2, 35);"""
+ refreshIncremental("test_ivm_expr_over_agg_3_array_agg")
+ refreshIncremental("test_ivm_expr_over_agg_3_collect_list")
+ refreshIncremental("test_ivm_expr_over_agg_3_bitmap_union")
+ refreshIncremental("test_ivm_expr_over_agg_3_bitmap_union_count")
+ order_qt_after_insert_array_agg """SELECT k, lst FROM test_ivm_expr_over_agg_3_array_agg"""
+ order_qt_after_insert_array_agg_source """
+ SELECT k, ARRAY_SORT(ARRAY_AGG(v)) AS lst FROM test_ivm_expr_over_agg_3_base GROUP BY k"""
+ order_qt_after_insert_collect_list """SELECT k, lst FROM test_ivm_expr_over_agg_3_collect_list"""
+ order_qt_after_insert_collect_list_source """
+ SELECT k, ARRAY_SORT(COLLECT_LIST(v)) AS lst FROM test_ivm_expr_over_agg_3_base GROUP BY k"""
+ order_qt_after_insert_bitmap_union """SELECT k, b FROM test_ivm_expr_over_agg_3_bitmap_union"""
+ order_qt_after_insert_bitmap_union_source """
+ SELECT k, BITMAP_COUNT(BITMAP_UNION(TO_BITMAP(v))) AS b FROM test_ivm_expr_over_agg_3_base GROUP BY k"""
+ order_qt_after_insert_bitmap_union_count """SELECT k, b FROM test_ivm_expr_over_agg_3_bitmap_union_count"""
+ order_qt_after_insert_bitmap_union_count_source """
+ SELECT k, BITMAP_UNION_COUNT(TO_BITMAP(v)) + 0 AS b FROM test_ivm_expr_over_agg_3_base GROUP BY k"""
+
+ // Equal-count update: the array contents change without the element count changing.
+ sql """INSERT INTO test_ivm_expr_over_agg_3_base VALUES (2, 1, 25);"""
+ refreshIncremental("test_ivm_expr_over_agg_3_array_agg")
+ refreshIncremental("test_ivm_expr_over_agg_3_collect_list")
+ order_qt_after_update_array_agg """SELECT k, lst FROM test_ivm_expr_over_agg_3_array_agg"""
+ order_qt_after_update_array_agg_source """
+ SELECT k, ARRAY_SORT(ARRAY_AGG(v)) AS lst FROM test_ivm_expr_over_agg_3_base GROUP BY k"""
+ order_qt_after_update_collect_list """SELECT k, lst FROM test_ivm_expr_over_agg_3_collect_list"""
+ order_qt_after_update_collect_list_source """
+ SELECT k, ARRAY_SORT(COLLECT_LIST(v)) AS lst FROM test_ivm_expr_over_agg_3_base GROUP BY k"""
+
+ // Delete the middle value of k=2 and add one to k=1, then refresh the arrays again.
+ sql """DELETE FROM test_ivm_expr_over_agg_3_base WHERE id = 7;"""
+ sql """INSERT INTO test_ivm_expr_over_agg_3_base VALUES (8, 1, 40);"""
+ refreshIncremental("test_ivm_expr_over_agg_3_array_agg")
+ refreshIncremental("test_ivm_expr_over_agg_3_collect_list")
+ order_qt_after_delete_array_agg """SELECT k, lst FROM test_ivm_expr_over_agg_3_array_agg"""
+ order_qt_after_delete_array_agg_source """
+ SELECT k, ARRAY_SORT(ARRAY_AGG(v)) AS lst FROM test_ivm_expr_over_agg_3_base GROUP BY k"""
+ order_qt_after_delete_collect_list """SELECT k, lst FROM test_ivm_expr_over_agg_3_collect_list"""
+ order_qt_after_delete_collect_list_source """
+ SELECT k, ARRAY_SORT(COLLECT_LIST(v)) AS lst FROM test_ivm_expr_over_agg_3_base GROUP BY k"""
+
+ // Every view is also completely refreshable and agrees with its source query.
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_3_array_agg COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_3_array_agg")
+ order_qt_complete_array_agg """SELECT k, lst FROM test_ivm_expr_over_agg_3_array_agg"""
+ order_qt_complete_array_agg_source """
+ SELECT k, ARRAY_SORT(ARRAY_AGG(v)) AS lst FROM test_ivm_expr_over_agg_3_base GROUP BY k"""
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_3_collect_list COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_3_collect_list")
+ order_qt_complete_collect_list """SELECT k, lst FROM test_ivm_expr_over_agg_3_collect_list"""
+ order_qt_complete_collect_list_source """
+ SELECT k, ARRAY_SORT(COLLECT_LIST(v)) AS lst FROM test_ivm_expr_over_agg_3_base GROUP BY k"""
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_3_bitmap_union COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_3_bitmap_union")
+ order_qt_complete_bitmap_union """SELECT k, b FROM test_ivm_expr_over_agg_3_bitmap_union"""
+ order_qt_complete_bitmap_union_source """
+ SELECT k, BITMAP_COUNT(BITMAP_UNION(TO_BITMAP(v))) AS b FROM test_ivm_expr_over_agg_3_base GROUP BY k"""
+ sql """REFRESH MATERIALIZED VIEW test_ivm_expr_over_agg_3_bitmap_union_count COMPLETE"""
+ waitingMTMVTaskFinishedByMvName("test_ivm_expr_over_agg_3_bitmap_union_count")
+ order_qt_complete_bitmap_union_count """SELECT k, b FROM test_ivm_expr_over_agg_3_bitmap_union_count"""
+ order_qt_complete_bitmap_union_count_source """
+ SELECT k, BITMAP_UNION_COUNT(TO_BITMAP(v)) + 0 AS b FROM test_ivm_expr_over_agg_3_base GROUP BY k"""
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org