You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by bo...@apache.org on 2024/05/25 08:08:58 UTC
(impala) branch master updated (b975165a0 -> d0237fbe4)
This is an automated email from the ASF dual-hosted git repository.
boroknagyz pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
from b975165a0 IMPALA-13034: Add logs and counters for HTTP profile requests blocking client fetches
new 8a6f2824b IMPALA-13105: Fix multiple imported query profiles fail to import/clear at once
new d0237fbe4 IMPALA-8042: Assign BETWEEN selectivity for discrete-unique column
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../java/org/apache/impala/analysis/Analyzer.java | 28 +-
.../apache/impala/analysis/BinaryPredicate.java | 18 +
.../apache/impala/analysis/CompoundPredicate.java | 34 +-
.../main/java/org/apache/impala/analysis/Expr.java | 26 +-
.../main/java/org/apache/impala/catalog/Type.java | 1 +
.../java/org/apache/impala/planner/PlanNode.java | 59 +-
.../impala/rewrite/BetweenToCompoundRule.java | 80 +-
.../impala/analysis/ExprCardinalityTest.java | 46 +-
testdata/bin/compute-table-stats.sh | 2 +
.../functional/functional_schema_template.sql | 15 +
.../datasets/functional/schema_constraints.csv | 4 +
.../queries/PlannerTest/card-scan.test | 172 ++-
.../PlannerTest/tpcds_cpu_cost/tpcds-q05.test | 524 ++++-----
.../PlannerTest/tpcds_cpu_cost/tpcds-q12.test | 370 +++---
.../PlannerTest/tpcds_cpu_cost/tpcds-q16.test | 362 +++---
.../PlannerTest/tpcds_cpu_cost/tpcds-q20.test | 370 +++---
.../PlannerTest/tpcds_cpu_cost/tpcds-q21.test | 298 ++---
.../PlannerTest/tpcds_cpu_cost/tpcds-q32.test | 204 ++--
.../PlannerTest/tpcds_cpu_cost/tpcds-q37.test | 96 +-
.../PlannerTest/tpcds_cpu_cost/tpcds-q40.test | 394 +++----
.../PlannerTest/tpcds_cpu_cost/tpcds-q77.test | 906 +++++++--------
.../PlannerTest/tpcds_cpu_cost/tpcds-q80.test | 1182 ++++++++++----------
.../PlannerTest/tpcds_cpu_cost/tpcds-q82.test | 96 +-
.../PlannerTest/tpcds_cpu_cost/tpcds-q92.test | 661 ++++++-----
.../PlannerTest/tpcds_cpu_cost/tpcds-q94.test | 362 +++---
.../PlannerTest/tpcds_cpu_cost/tpcds-q95.test | 350 +++---
.../PlannerTest/tpcds_cpu_cost/tpcds-q98.test | 386 +++----
www/queries.tmpl | 7 +-
28 files changed, 3730 insertions(+), 3323 deletions(-)
(impala) 01/02: IMPALA-13105: Fix multiple imported query profiles fail to import/clear at once
Posted by bo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
boroknagyz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 8a6f2824b8abf53ea022ca571da33619d564a14a
Author: Surya Hebbar <su...@protonmail.com>
AuthorDate: Fri May 10 15:58:47 2024 +0530
IMPALA-13105: Fix multiple imported query profiles fail to import/clear at once
On importing multiple query profiles, insertion of the last query in the
queue fails as no delay is provided for the insertion.
This has been fixed by providing a delay after inserting the final query.
On clearing all the imported queries, in some instances page reloads
before clearing the IndexedDB object store.
This has been fixed by triggering the page reload after clearing
the object store succeeds.
Change-Id: I42470fecd0cff6e193f080102575e51d86a2d562
Reviewed-on: http://gerrit.cloudera.org:8080/21450
Reviewed-by: Wenzhe Zhou <wz...@cloudera.com>
Reviewed-by: Riza Suminto <ri...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
www/queries.tmpl | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/www/queries.tmpl b/www/queries.tmpl
index 5422989d2..666d93767 100644
--- a/www/queries.tmpl
+++ b/www/queries.tmpl
@@ -282,8 +282,9 @@ command line parameter.</p>
}
function clearProfiles() {
- db.transaction("profiles", "readwrite").objectStore("profiles").clear();
- setScrollReload();
+ db.transaction("profiles", "readwrite").objectStore("profiles").clear().onsuccess = () => {
+ setScrollReload();
+ }
}
function uploadProfile() {
@@ -330,7 +331,7 @@ command line parameter.</p>
profileStore.put(query).onsuccess = () => {
uploadCount++;
if (uploadCount == json_profile_chooser.files.length) {
- setScrollReload();
+ setTimeout(setScrollReload, 1000);
}
};
} catch (err) {
(impala) 02/02: IMPALA-8042: Assign BETWEEN selectivity for discrete-unique column
Posted by bo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
boroknagyz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit d0237fbe47eb5089ee19a0a201045b862d65ecaa
Author: Riza Suminto <ri...@cloudera.com>
AuthorDate: Mon Apr 15 22:26:01 2024 -0700
IMPALA-8042: Assign BETWEEN selectivity for discrete-unique column
Impala frontend can not evaluate BETWEEN/NOT BETWEEN predicate directly.
It needs to transform a BetweenPredicate into a CompoundPredicate
consisting of upper bound and lower bound BinaryPredicate through
BetweenToCompoundRule.java. The BinaryPredicate can then be pushed down
or rewritten into other form by another expression rewrite rule.
However, the selectivity of BetweenPredicate or its derivatives remains
unassigned and often collapses with other unknown selectivity predicates
to have collective selectivity equals Expr.DEFAULT_SELECTIVITY (0.1).
This patch adds a narrow optimization of BetweenPredicate selectivity
when the following criteria are met:
1. The BetweenPredicate is bound to a slot reference of a single column
of a table.
2. The column type is discrete, such as INTEGER or DATE.
3. The column stats are available.
4. The column is sufficiently unique based on available stats.
5. The BETWEEN/NOT BETWEEN predicate is in good form (lower bound value
<= upper bound value).
6. The final calculated selectivity is less than or equal to
Expr.DEFAULT_SELECTIVITY.
If these criteria are unmet, the Planner will revert to the old
behavior, which is letting the selectivity unassigned.
Since this patch only target BetweenPredicate over unique column, the
following query will still have the default scan selectivity (0.1):
select count(*) from tpch.customer c
where c.c_custkey >= 1234 and c.c_custkey <= 2345;
While this equivalent query written with BETWEEN predicate will have
lower scan selectivity:
select count(*) from tpch.customer c
where c.c_custkey between 1234 and 2345;
This patch calculates the BetweenPredicate selectivity during
transformation at BetweenToCompoundRule.java. The selectivity is
piggy-backed into the resulting CompoundPredicate and BinaryPredicate as
betweenSelectivity_ field, separate from the selectivity_ field.
Analyzer.getBoundPredicates() is modified to prioritize the derived
BinaryPredicate over ordinary BinaryPredicate in its return value to
prevent the derived BinaryPredicate from being eliminated by a matching
ordinary BinaryPredicate.
Testing:
- Add table functional_parquet.unique_with_nulls.
- Add FE tests in ExprCardinalityTest#testBetweenSelectivity,
ExprCardinalityTest#testNotBetweenSelectivity, and
PlannerTest#testScanCardinality.
- Pass core tests.
Change-Id: Ib349d97349d1ee99788645a66be1b81749684d10
Reviewed-on: http://gerrit.cloudera.org:8080/21377
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
.../java/org/apache/impala/analysis/Analyzer.java | 28 +-
.../apache/impala/analysis/BinaryPredicate.java | 18 +
.../apache/impala/analysis/CompoundPredicate.java | 34 +-
.../main/java/org/apache/impala/analysis/Expr.java | 26 +-
.../main/java/org/apache/impala/catalog/Type.java | 1 +
.../java/org/apache/impala/planner/PlanNode.java | 59 +-
.../impala/rewrite/BetweenToCompoundRule.java | 80 +-
.../impala/analysis/ExprCardinalityTest.java | 46 +-
testdata/bin/compute-table-stats.sh | 2 +
.../functional/functional_schema_template.sql | 15 +
.../datasets/functional/schema_constraints.csv | 4 +
.../queries/PlannerTest/card-scan.test | 172 ++-
.../PlannerTest/tpcds_cpu_cost/tpcds-q05.test | 524 ++++-----
.../PlannerTest/tpcds_cpu_cost/tpcds-q12.test | 370 +++---
.../PlannerTest/tpcds_cpu_cost/tpcds-q16.test | 362 +++---
.../PlannerTest/tpcds_cpu_cost/tpcds-q20.test | 370 +++---
.../PlannerTest/tpcds_cpu_cost/tpcds-q21.test | 298 ++---
.../PlannerTest/tpcds_cpu_cost/tpcds-q32.test | 204 ++--
.../PlannerTest/tpcds_cpu_cost/tpcds-q37.test | 96 +-
.../PlannerTest/tpcds_cpu_cost/tpcds-q40.test | 394 +++----
.../PlannerTest/tpcds_cpu_cost/tpcds-q77.test | 906 +++++++--------
.../PlannerTest/tpcds_cpu_cost/tpcds-q80.test | 1182 ++++++++++----------
.../PlannerTest/tpcds_cpu_cost/tpcds-q82.test | 96 +-
.../PlannerTest/tpcds_cpu_cost/tpcds-q92.test | 661 ++++++-----
.../PlannerTest/tpcds_cpu_cost/tpcds-q94.test | 362 +++---
.../PlannerTest/tpcds_cpu_cost/tpcds-q95.test | 350 +++---
.../PlannerTest/tpcds_cpu_cost/tpcds-q98.test | 386 +++----
27 files changed, 3726 insertions(+), 3320 deletions(-)
diff --git a/fe/src/main/java/org/apache/impala/analysis/Analyzer.java b/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
index 65a7c774a..2590236b4 100644
--- a/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
+++ b/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
@@ -2460,6 +2460,8 @@ public class Analyzer {
*/
public List<Expr> getBoundPredicates(TupleId destTid, Set<SlotId> ignoreSlots,
boolean markAssigned) {
+ // Map that tracks BinaryPredicates that derived from the same BetweenPredicate.
+ Map<ExprId, List<BinaryPredicate>> betweenPredicates = new HashMap<>();
List<Expr> result = new ArrayList<>();
for (ExprId srcConjunctId: globalState_.singleTidConjuncts) {
Expr srcConjunct = globalState_.conjuncts.get(srcConjunctId);
@@ -2610,9 +2612,31 @@ public class Analyzer {
}
}
- // check if we already created this predicate
- if (!result.contains(p)) result.add(p);
+ if ((p instanceof BinaryPredicate)
+ && ((BinaryPredicate) p).derivedFromBetween()) {
+ BinaryPredicate b = (BinaryPredicate) p;
+ betweenPredicates.computeIfAbsent(b.getBetweenExprId(), k -> new ArrayList<>());
+ betweenPredicates.get(b.getBetweenExprId()).add(b);
+ } else {
+ // check if we already created this predicate
+ if (!result.contains(p)) result.add(p);
+ }
+ }
+ }
+
+ if (!betweenPredicates.isEmpty()) {
+ // Prioritize members of 'betweenPredicates' ahead of 'result'.
+ // BinaryPredicates that derived from BetweenPredicates may have lower selectivity
+ // estimate from BetweenToCompoundRule. Placing them in-front will ensure that
+ // they are retained over other matching BinaryPredicates that do not come from
+ // BetweenPredicates when passed through Expr.removeDuplicates().
+ List<Expr> prioritizedExprs = new ArrayList<>();
+ for (List<BinaryPredicate> predicates : betweenPredicates.values()) {
+ prioritizedExprs.addAll(predicates);
}
+ prioritizedExprs.addAll(result);
+ Expr.removeDuplicates(prioritizedExprs);
+ result = prioritizedExprs;
}
return result;
}
diff --git a/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java b/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java
index 21b600055..9d99b98e9 100644
--- a/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java
+++ b/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java
@@ -46,6 +46,9 @@ public class BinaryPredicate extends Predicate {
// true if this BinaryPredicate is inferred from slot equivalences, false otherwise.
private boolean isInferred_ = false;
+ private ExprId betweenExprId_ = null;
+ private double betweenSelectivity_ = -1;
+
public enum Operator {
EQ("=", "eq", TComparisonOp.EQ),
NE("!=", "ne", TComparisonOp.NE),
@@ -155,6 +158,8 @@ public class BinaryPredicate extends Predicate {
super(other);
op_ = other.op_;
isInferred_ = other.isInferred_;
+ betweenExprId_ = other.betweenExprId_;
+ betweenSelectivity_ = other.betweenSelectivity_;
}
public boolean isNullMatchingEq() { return op_ == Operator.NULL_MATCHING_EQ; }
@@ -193,6 +198,10 @@ public class BinaryPredicate extends Predicate {
toStrHelper.add("op", op_).addValue(super.debugString());
if (isAuxExpr()) toStrHelper.add("isAux", true);
if (isInferred_) toStrHelper.add("isInferred", true);
+ if (derivedFromBetween()) {
+ toStrHelper.add("betweenExprId", betweenExprId_);
+ toStrHelper.add("betweenSelectivity", betweenSelectivity_);
+ }
return toStrHelper.toString();
}
@@ -415,4 +424,13 @@ public class BinaryPredicate extends Predicate {
@Override
public Expr clone() { return new BinaryPredicate(this); }
+
+ public void setBetweenSelectivity(ExprId betweenExprId, double betweenSelectivity) {
+ betweenExprId_ = betweenExprId;
+ betweenSelectivity_ = betweenSelectivity;
+ }
+
+ public boolean derivedFromBetween() { return betweenExprId_ != null; }
+ public ExprId getBetweenExprId() { return betweenExprId_; }
+ public double getBetweenSelectivity() { return betweenSelectivity_; }
}
diff --git a/fe/src/main/java/org/apache/impala/analysis/CompoundPredicate.java b/fe/src/main/java/org/apache/impala/analysis/CompoundPredicate.java
index cc169a183..5a31f923d 100644
--- a/fe/src/main/java/org/apache/impala/analysis/CompoundPredicate.java
+++ b/fe/src/main/java/org/apache/impala/analysis/CompoundPredicate.java
@@ -28,6 +28,7 @@ import org.apache.impala.thrift.TExprNode;
import org.apache.impala.thrift.TExprNodeType;
import com.google.common.base.MoreObjects;
+import com.google.common.base.MoreObjects.ToStringHelper;
import com.google.common.base.Preconditions;
import com.google.common.collect.Lists;
@@ -54,6 +55,11 @@ public class CompoundPredicate extends Predicate {
}
private final Operator op_;
+ // Selectivity estimate from BetweenToCompoundRule.computeBetweenSelectivity().
+ // Only set if this CompoundPredicate is a rewrite from BetweenPredicate.
+ // Otherwise, set to -1.
+ private final double betweenSelectivity_;
+
public static void initBuiltins(Db db) {
// AND and OR are implemented as custom exprs, so they do not have a function symbol.
db.addBuiltin(ScalarFunction.createBuiltinOperator(
@@ -67,7 +73,13 @@ public class CompoundPredicate extends Predicate {
Lists.<Type>newArrayList(Type.BOOLEAN), Type.BOOLEAN));
}
- public CompoundPredicate(Operator op, Expr e1, Expr e2) {
+ public static CompoundPredicate createFromBetweenPredicate(Operator op,
+ BinaryPredicate lower, BinaryPredicate upper, double betweenSelectivity) {
+ Preconditions.checkArgument(betweenSelectivity >= 0.0);
+ return new CompoundPredicate(op, lower, upper, betweenSelectivity);
+ }
+
+ private CompoundPredicate(Operator op, Expr e1, Expr e2, double betweenSelectivity) {
super();
this.op_ = op;
Preconditions.checkNotNull(e1);
@@ -84,14 +96,18 @@ public class CompoundPredicate extends Predicate {
setHasAlwaysTrueHint(true);
}
}
+ betweenSelectivity_ = betweenSelectivity;
}
+ public CompoundPredicate(Operator op, Expr e1, Expr e2) { this(op, e1, e2, -1); }
+
/**
* Copy c'tor used in clone().
*/
protected CompoundPredicate(CompoundPredicate other) {
super(other);
op_ = other.op_;
+ betweenSelectivity_ = other.betweenSelectivity_;
}
public Operator getOp() { return op_; }
@@ -103,10 +119,11 @@ public class CompoundPredicate extends Predicate {
@Override
public String debugString() {
- return MoreObjects.toStringHelper(this)
+ ToStringHelper helper = MoreObjects.toStringHelper(this)
.add("op", op_)
- .addValue(super.debugString())
- .toString();
+ .addValue(super.debugString());
+ if (betweenSelectivity_ != -1) helper.add("betweenSelectivity", betweenSelectivity_);
+ return helper.toString();
}
@Override
@@ -163,8 +180,11 @@ public class CompoundPredicate extends Predicate {
@Deprecated
protected void computeSelectivity() {
- if (!getChild(0).hasSelectivity() ||
- (children_.size() == 2 && !getChild(1).hasSelectivity())) {
+ if (betweenSelectivity_ != -1) {
+ selectivity_ = betweenSelectivity_;
+ return;
+ } else if (!getChild(0).hasSelectivity()
+ || (children_.size() == 2 && !getChild(1).hasSelectivity())) {
// Give up if one of our children has an unknown selectivity.
selectivity_ = -1;
return;
@@ -238,4 +258,6 @@ public class CompoundPredicate extends Predicate {
if (rhs == null) return lhs;
return new CompoundPredicate(Operator.AND, rhs, lhs);
}
+
+ public boolean derivedFromBetween() { return betweenSelectivity_ != -1; }
}
diff --git a/fe/src/main/java/org/apache/impala/analysis/Expr.java b/fe/src/main/java/org/apache/impala/analysis/Expr.java
index c5e3f02de..4a07e67a6 100644
--- a/fe/src/main/java/org/apache/impala/analysis/Expr.java
+++ b/fe/src/main/java/org/apache/impala/analysis/Expr.java
@@ -1851,9 +1851,11 @@ abstract public class Expr extends TreeNode<Expr> implements ParseNode, Cloneabl
* Analyzes and evaluates expression to an integral value, returned as a long.
* Throws if the expression cannot be evaluated or if the value evaluates to null.
* The 'name' parameter is used in exception messages, e.g. "LIMIT expression
- * evaluates to NULL".
+ * evaluates to NULL". If 'acceptDate' is true, treat Date expression as integral
+ * expression as well.
*/
- public long evalToInteger(Analyzer analyzer, String name) throws AnalysisException {
+ public long evalToInteger(Analyzer analyzer, String name, boolean acceptDate)
+ throws AnalysisException {
// Check for slotrefs and subqueries before analysis so we can provide a more
// helpful error message.
if (contains(SlotRef.class) || contains(Subquery.class)) {
@@ -1865,7 +1867,7 @@ abstract public class Expr extends TreeNode<Expr> implements ParseNode, Cloneabl
throw new AnalysisException(name + " expression must be a constant expression: " +
toSql());
}
- if (!getType().isIntegerType()) {
+ if (!getType().isIntegerType() && !(acceptDate && getType().isDate())) {
throw new AnalysisException(name + " expression must be an integer type but is '" +
getType() + "': " + toSql());
}
@@ -1875,6 +1877,16 @@ abstract public class Expr extends TreeNode<Expr> implements ParseNode, Cloneabl
} catch (InternalException e) {
throw new AnalysisException("Failed to evaluate expr: " + toSql(), e);
}
+
+ try {
+ return evalToInteger(val, acceptDate);
+ } catch (AnalysisException e) {
+ throw new AnalysisException(name + " expression evaluates to NULL: " + toSql());
+ }
+ }
+
+ public static long evalToInteger(TColumnValue val, boolean acceptDate)
+ throws AnalysisException {
long value;
if (val.isSetLong_val()) {
value = val.getLong_val();
@@ -1884,12 +1896,18 @@ abstract public class Expr extends TreeNode<Expr> implements ParseNode, Cloneabl
value = val.getShort_val();
} else if (val.isSetByte_val()) {
value = val.getByte_val();
+ } else if (acceptDate && val.isSetDate_val()) {
+ value = val.getDate_val();
} else {
- throw new AnalysisException(name + " expression evaluates to NULL: " + toSql());
+ throw new AnalysisException("TColumnValue evaluates to NULL: " + val);
}
return value;
}
+ public long evalToInteger(Analyzer analyzer, String name) throws AnalysisException {
+ return evalToInteger(analyzer, name, false);
+ }
+
/**
* Analyzes and evaluates expression to a non-negative integral value, returned as a
* long. Throws if the expression cannot be evaluated, if the value evaluates to null,
diff --git a/fe/src/main/java/org/apache/impala/catalog/Type.java b/fe/src/main/java/org/apache/impala/catalog/Type.java
index 98c051da8..40f4f53f5 100644
--- a/fe/src/main/java/org/apache/impala/catalog/Type.java
+++ b/fe/src/main/java/org/apache/impala/catalog/Type.java
@@ -234,6 +234,7 @@ public abstract class Type {
|| isScalarType(PrimitiveType.TIMESTAMP);
}
+ public boolean isIntegerOrDateType() { return isIntegerType() || isDate(); }
public boolean isComplexType() { return isStructType() || isCollectionType(); }
public boolean isCollectionType() { return isMapType() || isArrayType(); }
public boolean isMapType() { return this instanceof MapType; }
diff --git a/fe/src/main/java/org/apache/impala/planner/PlanNode.java b/fe/src/main/java/org/apache/impala/planner/PlanNode.java
index 7054e21fc..dbc7b74b4 100644
--- a/fe/src/main/java/org/apache/impala/planner/PlanNode.java
+++ b/fe/src/main/java/org/apache/impala/planner/PlanNode.java
@@ -21,18 +21,22 @@ import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
-import java.util.Iterator;
+import java.util.HashMap;
import java.util.HashSet;
+import java.util.Iterator;
import java.util.List;
+import java.util.Map;
import java.util.Set;
import java.util.Stack;
import org.apache.impala.analysis.Analyzer;
import org.apache.impala.analysis.BinaryPredicate;
+import org.apache.impala.analysis.CompoundPredicate;
import org.apache.impala.analysis.Expr;
import org.apache.impala.analysis.ExprId;
import org.apache.impala.analysis.ExprSubstitutionMap;
import org.apache.impala.analysis.SlotDescriptor;
+import org.apache.impala.analysis.SlotId;
import org.apache.impala.analysis.SlotRef;
import org.apache.impala.analysis.ToSqlOptions;
import org.apache.impala.analysis.TupleDescriptor;
@@ -751,19 +755,59 @@ abstract public class PlanNode extends TreeNode<PlanNode> {
* 1. The individual selectivities of conjuncts may be unknown.
* 2. Two selectivities, whether known or unknown, could be correlated. Assuming
* independence can lead to significant underestimation.
+ * 3. Two BinaryPredicate may be derived from single BetweenPredicate that have
+ * lower selectivity if analyzed as a pair.
*
* The first issue is addressed by using a single default selectivity that is
* representative of all conjuncts with unknown selectivities.
* The second issue is addressed by an exponential backoff when multiplying each
* additional selectivity into the final result.
+ * The third issue is addressed by checking BinaryPredicate.derivedFromBetween()
+ * property. If it is True, calculate the expression selectivity from
+ * BinaryPredicate.getBetweenSelectivity(). Both of the BinaryPredicates should have
+ * the same getBetweenSelectivity() value, thus it is OK to inspect just from one of
+ * them.
+ * TODO: Fix the third issue with more general solution when there are multiple range
+ * predicates involved, even if some of them are not derived from BetweenPredicate.
*/
static protected double computeCombinedSelectivity(List<Expr> conjuncts) {
// Collect all estimated selectivities.
List<Double> selectivities = new ArrayList<>();
+ // Map between a slot id and the lowest between selectivity targeting that slot.
+ Map<SlotId, Double> perSlotBetweenSelectivities = new HashMap<>();
+
+ int conjunctSize = 0;
for (Expr e: conjuncts) {
- if (e.hasSelectivity()) selectivities.add(e.getSelectivity());
+ if (e instanceof BinaryPredicate && ((BinaryPredicate) e).derivedFromBetween()) {
+ // This is one of two BinaryPredicate that derived from a BetweenPredicate.
+ // A pair of BetweenPredicate must have been assigned and pushed down to the same
+ // 'targetSlotId', or one of them maybe removed by Expr.removeDuplicates()
+ // somewhere by Analyzer. But it is enough to get the selectivity from one of
+ // them. Analyzer.getBoundPredicates() ensure that BinaryPredicate derived from
+ // BetweenPredicate gets priority over regular BinaryPredicate.
+ // If multiple BetweenPredicate target the same 'targetSlotId', pick the least
+ // selectivity.
+ BinaryPredicate pred = (BinaryPredicate) e;
+ insertBetweenSelectivity(pred, perSlotBetweenSelectivities);
+ } else if (e instanceof CompoundPredicate
+ && ((CompoundPredicate) e).derivedFromBetween()
+ && (((CompoundPredicate) e).getChild(0) instanceof BinaryPredicate)) {
+ BinaryPredicate pred = (BinaryPredicate) ((CompoundPredicate) e).getChild(0);
+ insertBetweenSelectivity(pred, perSlotBetweenSelectivities);
+ } else {
+ // For everything else.
+ if (e.hasSelectivity()) selectivities.add(e.getSelectivity());
+ conjunctSize++;
+ }
}
- if (selectivities.size() != conjuncts.size()) {
+
+ // Add all values from perSlotBetweenSelectivities to selectivities.
+ if (!perSlotBetweenSelectivities.isEmpty()) {
+ selectivities.addAll(perSlotBetweenSelectivities.values());
+ conjunctSize += perSlotBetweenSelectivities.size();
+ }
+
+ if (selectivities.size() != conjunctSize) {
// Some conjuncts have no estimated selectivity. Use a single default
// representative selectivity for all those conjuncts.
selectivities.add(Expr.DEFAULT_SELECTIVITY);
@@ -781,6 +825,15 @@ abstract public class PlanNode extends TreeNode<PlanNode> {
return Math.max(0.0, Math.min(1.0, result));
}
+ static private void insertBetweenSelectivity(
+ BinaryPredicate pred, Map<SlotId, Double> perSlotBetweenSelectivities) {
+ Preconditions.checkNotNull(pred.getBoundSlot());
+ Preconditions.checkState(pred.getBetweenSelectivity() >= 0.0);
+ SlotId targetSlotId = pred.getBoundSlot().getSlotId();
+ double sel = pred.getBetweenSelectivity();
+ perSlotBetweenSelectivities.merge(targetSlotId, sel, (v1, v2) -> Math.min(v1, v2));
+ }
+
protected double computeSelectivity() {
return computeCombinedSelectivity(conjuncts_);
}
diff --git a/fe/src/main/java/org/apache/impala/rewrite/BetweenToCompoundRule.java b/fe/src/main/java/org/apache/impala/rewrite/BetweenToCompoundRule.java
index bf2a39ce7..37bef1db0 100644
--- a/fe/src/main/java/org/apache/impala/rewrite/BetweenToCompoundRule.java
+++ b/fe/src/main/java/org/apache/impala/rewrite/BetweenToCompoundRule.java
@@ -22,7 +22,13 @@ import org.apache.impala.analysis.BetweenPredicate;
import org.apache.impala.analysis.BinaryPredicate;
import org.apache.impala.analysis.CompoundPredicate;
import org.apache.impala.analysis.Expr;
-import org.apache.impala.analysis.Predicate;
+import org.apache.impala.analysis.SlotDescriptor;
+import org.apache.impala.analysis.SlotRef;
+import org.apache.impala.catalog.ColumnStats;
+import org.apache.impala.catalog.FeTable;
+import org.apache.impala.common.AnalysisException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
/**
* Rewrites BetweenPredicates into an equivalent conjunctive/disjunctive
@@ -34,6 +40,7 @@ import org.apache.impala.analysis.Predicate;
* A NOT BETWEEN X AND Y ==> A < X OR A > Y
*/
public class BetweenToCompoundRule implements ExprRewriteRule {
+ private final static Logger LOG = LoggerFactory.getLogger(BetweenToCompoundRule.class);
public static final ExprRewriteRule INSTANCE = new BetweenToCompoundRule();
@Override
@@ -62,9 +69,74 @@ public class BetweenToCompoundRule implements ExprRewriteRule {
upperOperator = BinaryPredicate.Operator.LE;
compoundOperator = CompoundPredicate.Operator.AND;
}
- Predicate lower = new BinaryPredicate(lowerOperator, value, lowerBound);
- Predicate upper = new BinaryPredicate(upperOperator, clonedValue, upperBound);
- return new CompoundPredicate(compoundOperator, lower, upper);
+ BinaryPredicate lower = new BinaryPredicate(lowerOperator, value, lowerBound);
+ BinaryPredicate upper = new BinaryPredicate(upperOperator, clonedValue, upperBound);
+ double sel = computeBetweenSelectivity(analyzer, compoundOperator, lower, upper);
+ CompoundPredicate pred;
+ if (sel > 0) {
+ lower.setBetweenSelectivity(bp.getId(), sel);
+ upper.setBetweenSelectivity(bp.getId(), sel);
+ pred = CompoundPredicate.createFromBetweenPredicate(
+ compoundOperator, lower, upper, sel);
+ } else {
+ pred = new CompoundPredicate(compoundOperator, lower, upper);
+ }
+
+ if (LOG.isTraceEnabled()) {
+ LOG.trace("Transformed " + bp.debugString() + " to " + pred.debugString());
+ }
+ return pred;
+ }
+
+ private static double computeBetweenSelectivity(Analyzer analyzer,
+ CompoundPredicate.Operator compoundOperator, BinaryPredicate lowerBound,
+ BinaryPredicate upperBound) {
+ SlotRef slotRef = lowerBound.getBoundSlot();
+ if (slotRef == null || !slotRef.hasDesc()) return -1;
+
+ // TODO: handle slotDesc that is a Union from different table columns.
+ SlotDescriptor slotDesc = slotRef.getDesc();
+ FeTable table = slotDesc.getParent().getTable();
+ if (table == null || table.getNumRows() < 0) return -1;
+ if (!slotDesc.getType().isIntegerOrDateType()) return -1;
+
+ ColumnStats stats = slotDesc.getStats();
+ if (!stats.hasNumDistinctValues()) return -1;
+
+ long numNotNulls = table.getNumRows() - Math.max(0, stats.getNumNulls());
+ if ((double) stats.getNumDistinctValues() / numNotNulls < 0.9) {
+ // column is not quite unique.
+ return -1;
+ }
+
+ try {
+ long upperVal = upperBound.getChild(1).evalToInteger(analyzer, "BETWEEN_up", true);
+ long lowerVal = lowerBound.getChild(1).evalToInteger(analyzer, "BETWEEN_low", true);
+
+ // If compute_column_minmax_stats=true, stats might have both high and low value
+ // set.
+ if (stats.getHighValue() != null) {
+ upperVal = Math.min(upperVal, Expr.evalToInteger(stats.getHighValue(), true));
+ }
+ if (stats.getLowValue() != null) {
+ lowerVal = Math.max(lowerVal, Expr.evalToInteger(stats.getLowValue(), true));
+ }
+ if (upperVal < lowerVal) return -1;
+
+ long diff = upperVal - lowerVal + 1;
+ double sel = Math.max(0.0, (double) diff / table.getNumRows());
+ if (compoundOperator == CompoundPredicate.Operator.OR) sel = 1.0 - sel;
+
+ // Only return selectivity if it is less than or equal to Expr.DEFAULT_SELECTIVITY.
+ // This is to maintain previous behavior where two BinaryPredicate derived from
+ // a BetweenPredicate will have unknown (-1) selectivity and
+ // PlanNode.computeCombinedSelectivity() will adds Expr.DEFAULT_SELECTIVITY to
+ // represent all conjunct with unknown selectivity.
+ if (sel <= Expr.DEFAULT_SELECTIVITY) return sel;
+ } catch (AnalysisException e) {
+ LOG.error("Error evaluating bounding value of BetweenPredicate", e);
+ }
+ return -1;
}
private BetweenToCompoundRule() {}
diff --git a/fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java b/fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java
index 3bd408a73..992aba752 100644
--- a/fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java
+++ b/fe/src/test/java/org/apache/impala/analysis/ExprCardinalityTest.java
@@ -219,16 +219,20 @@ public class ExprCardinalityTest {
verifySelectCol("manynulls", "id", -1, -1);
}
- public void verifySelectExpr(String table, String exprSql,
- long expectedNdv, double expectedSel) throws ImpalaException {
- SelectFixture select = new SelectFixture(session_)
- .table("functional." + table)
- .exprSql(exprSql);
+ public void verifySelectExpr(String db, String table, String exprSql, long expectedNdv,
+ double expectedSel) throws ImpalaException {
+ SelectFixture select =
+ new SelectFixture(session_).table(db + "." + table).exprSql(exprSql);
Expr expr = select.analyzeExpr();
assertEquals(expectedNdv, expr.getNumDistinctValues());
assertEquals(expectedSel, expr.getSelectivity(), 0.00001);
}
+ public void verifySelectExpr(String table, String exprSql, long expectedNdv,
+ double expectedSel) throws ImpalaException {
+ verifySelectExpr("functional", table, exprSql, expectedNdv, expectedSel);
+ }
+
/**
* Constants have an NDV of 1, selectivity of -1.
*/
@@ -535,20 +539,31 @@ public class ExprCardinalityTest {
/**
* Test col BETWEEN x and y. Rewritten to
* col >= x AND col <= y. Inequality should have an estimate. Since
- * the expression is an AND, we multipley the two estimates.
+ * the expression is an AND, we multiply the two estimates.
* So, regardless of NDV and null count, selectivity should be
* something like 0.33^2.
+ * However, for a sufficiently unique column, selectivity can be approximated as ratio
+ * of values that fall in that range over NDV
+ * (see BetweenToCompoundRule.computeBetweenSelectivity()).
*/
@Test
public void testBetweenSelectivity() throws ImpalaException {
// Bug: NO selectivity for Between because it is rewritten to
- // use inequalities, and there no selectivities for those
- // See IMPALA-8042
- //verifySelectExpr("alltypes", "id between 30 and 60", 3, 0.33 * 0.33);
- verifySelectExpr("alltypes", "id between 30 and 60", 3, -1);
+ // use inequalities, and there no selectivities for those, except for Between
+ // predicate over a sufficiently unique column. See IMPALA-8042
+ // verifySelectExpr("alltypes", "id between 30 and 60", 3, 0.33 * 0.33);
+ verifySelectExpr("alltypes", "id between 30 and 60", 3, 31.0 / 7300.0);
+ // Fallback to -1 if selectivity is too high.
+ verifySelectExpr("alltypes", "id between 1 and 7298", 3, -1);
//verifySelectExpr("alltypes", "int_col between 30 and 60", 3, 0.33 * 0.33);
verifySelectExpr("alltypes", "int_col between 30 and 60", 3, -1);
+ // Should work over date column.
+ // TODO: Add new unique date column in functional.date_tbl and use it here.
+ verifySelectExpr("tpcds_partitioned_parquet_snap", "date_dim",
+ "d_date between DATE '2001-03-13' and (DATE '2001-03-13' + interval 90 days)", 3,
+ 91.0 / 73049.0);
+
// Should not matter that there are no stats
//verifySelectExpr("manynulls", "id between 30 and 60", 3, 0.33 * 0.33);
verifySelectExpr("manynulls", "id between 30 and 60", 3, -1);
@@ -561,11 +576,20 @@ public class ExprCardinalityTest {
public void testNotBetweenSelectivity() throws ImpalaException {
// Bug: NO selectivity for Not Between because it is rewritten to
// use inequalities, and there no selectivities for those
- //verifySelectExpr("alltypes", "id not between 30 and 60", 3, 1 - 0.33 * 0.33);
+ // verifySelectExpr("alltypes", "id not between 30 and 60", 3, 1 - 0.33 * 0.33);
+ // Fallback to -1 if selectivity is too high.
verifySelectExpr("alltypes", "id not between 30 and 60", 3, -1);
+ // only 2 value not in between.
+ verifySelectExpr("alltypes", "id not between 1 and 7298", 3, 2.0 / 7300.0);
//verifySelectExpr("alltypes", "int_col not between 30 and 60", 3, 1 - 0.33 * 0.33);
verifySelectExpr("alltypes", "int_col not between 30 and 60", 3, -1);
+ // Should work over date column. Fallback to -1 if selectivity is too high.
+ // TODO: Add new unique date column in functional.date_tbl and use it here.
+ verifySelectExpr("tpcds_partitioned_parquet_snap", "date_dim",
+ "d_date not between DATE '2001-03-13' and (DATE '2001-03-13' + interval 90 days)",
+ 3, -1);
+
// Should not matter that there are no stats
//verifySelectExpr("manynulls", "id not between 30 and 60", 3, 1 - 0.33 * 0.33);
verifySelectExpr("manynulls", "id not between 30 and 60", 3, -1);
diff --git a/testdata/bin/compute-table-stats.sh b/testdata/bin/compute-table-stats.sh
index 7ca914319..40001eaa4 100755
--- a/testdata/bin/compute-table-stats.sh
+++ b/testdata/bin/compute-table-stats.sh
@@ -36,6 +36,8 @@ ${COMPUTE_STATS_SCRIPT} --db_names=functional\
alltypessmall,alltypestiny,jointbl,dimtbl,stringpartitionkey,nulltable,nullrows,
date_tbl,chars_medium,part_strings_with_quotes,alltypes_date_partition,
alltypes_date_partition_2,mv1_alltypes_jointbl,binary_tbl,binary_tbl_big"
+${COMPUTE_STATS_SCRIPT} --db_names=functional_parquet \
+ --table_names="unique_with_nulls"
# We cannot load HBase on s3 and isilon yet.
if [ "${TARGET_FILESYSTEM}" = "hdfs" ]; then
diff --git a/testdata/datasets/functional/functional_schema_template.sql b/testdata/datasets/functional/functional_schema_template.sql
index 762f6e319..9f4c2aa67 100644
--- a/testdata/datasets/functional/functional_schema_template.sql
+++ b/testdata/datasets/functional/functional_schema_template.sql
@@ -4705,3 +4705,18 @@ PARTITION BY HASH (id) PARTITIONS 3 STORED AS KUDU;
INSERT into TABLE {db_name}{db_suffix}.{table_name}
SELECT * FROM {db_name}.{table_name};
====
+---- DATASET
+functional
+---- BASE_TABLE_NAME
+unique_with_nulls
+---- COLUMNS
+id int
+int_col int
+date_col date
+---- DEPENDENT_LOAD
+INSERT OVERWRITE TABLE {db_name}{db_suffix}.{table_name}
+SELECT id,
+ case when id % 2 = 0 then id else null end,
+ case when id % 2 = 0 then date_add(DATE '2023-12-31', interval id days) else null end
+FROM functional.alltypessmall order by id;
+====
diff --git a/testdata/datasets/functional/schema_constraints.csv b/testdata/datasets/functional/schema_constraints.csv
index 58b705e78..035fccf41 100644
--- a/testdata/datasets/functional/schema_constraints.csv
+++ b/testdata/datasets/functional/schema_constraints.csv
@@ -425,3 +425,7 @@ table_name:empty_stream_tbl, constraint:restrict_to, table_format:orc/def/block
# DST changes in the 'America/Los_Angeles' time zone.
table_name:timestamp_at_dst_changes, constraint:restrict_to, table_format:text/none/none
table_name:timestamp_at_dst_changes, constraint:restrict_to, table_format:kudu/none/none
+
+# Table where all column values are unique but have some NULLs.
+# Depends on functional.alltypessmall.
+table_name:unique_with_nulls, constraint:restrict_to, table_format:parquet/none/none
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/card-scan.test b/testdata/workloads/functional-planner/queries/PlannerTest/card-scan.test
index 20aee3946..e4eec0c6e 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/card-scan.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/card-scan.test
@@ -2,6 +2,9 @@
# In this file, "inequality" means not equals. It is a general
# term for <, <=, >, >=. The term "not equals" is used for the
# != (AKA <>) operator.
+# tpch.customer.c_custkey is a unique column with value range
+# [1..150000].
+#
# No predicate at all. Establishes cardinality baseline.
select * from tpch.customer
---- PLAN
@@ -242,12 +245,8 @@ PLAN-ROOT SINK
predicates: c.c_custkey < 1234, c.c_nationkey < 100
row-size=218B cardinality=15.00K
====
-# Between. No useful stats. Should assume, say 0.16
-# But, gets rewritten before computing selectivity,
-# So same as AND'ed inequalities
-# sel = 0.1, but Ramakrishnan and Gherke suggest 1/4
-# Bug: IMPALA-8037
-# Bug: Expected cardinality ~28.5K
+# Between over unique BIGINT column.
+# sel = 1112/150000
select *
from tpch.customer c
where c.c_custkey between 1234 and 2345
@@ -257,7 +256,7 @@ PLAN-ROOT SINK
00:SCAN HDFS [tpch.customer c]
HDFS partitions=1/1 files=1 size=23.08MB
predicates: c.c_custkey <= 2345, c.c_custkey >= 1234
- row-size=218B cardinality=15.00K
+ row-size=218B cardinality=1.11K
====
# Compound inequality (< and >) that is the same as BETWEEN.
# Gets compouted as AND of inequalities.
@@ -280,8 +279,33 @@ PLAN-ROOT SINK
predicates: c.c_custkey <= 2345, c.c_custkey >= 1234
row-size=218B cardinality=15.00K
====
-# Between and redundant inequality
-# Bug: expected cardinality ~15K
+# Between with wrong min and max bound value.
+# Default to 0.1 selectivity.
+select *
+from tpch.customer c
+where c.c_custkey between 2345 and 1234
+---- PLAN
+PLAN-ROOT SINK
+|
+00:SCAN HDFS [tpch.customer c]
+ HDFS partitions=1/1 files=1 size=23.08MB
+ predicates: c.c_custkey <= 1234, c.c_custkey >= 2345
+ row-size=218B cardinality=15.00K
+====
+# Between where both min and max bound exceed max value.
+# Bug: cardinality should be 0. Revisit this with compute_column_minmax_stats=true option.
+select *
+from tpch.customer c
+where c.c_custkey between 150001 and 151111
+---- PLAN
+PLAN-ROOT SINK
+|
+00:SCAN HDFS [tpch.customer c]
+ HDFS partitions=1/1 files=1 size=23.08MB
+ predicates: c.c_custkey <= 151111, c.c_custkey >= 150001
+ row-size=218B cardinality=1.11K
+====
+# Between and redundant inequality.
select *
from tpch.customer c
where c.c_custkey between 1234 and 2345
@@ -292,7 +316,135 @@ PLAN-ROOT SINK
00:SCAN HDFS [tpch.customer c]
HDFS partitions=1/1 files=1 size=23.08MB
predicates: c.c_custkey <= 2345, c.c_custkey >= 1234
- row-size=218B cardinality=15.00K
+ row-size=218B cardinality=1.11K
+====
+# Between and two redundant inequality
+select *
+from tpch.customer c
+where c.c_custkey between 1234 and 2345
+ and c.c_custkey <= 2345 and c.c_custkey >= 1234
+---- PLAN
+PLAN-ROOT SINK
+|
+00:SCAN HDFS [tpch.customer c]
+ HDFS partitions=1/1 files=1 size=23.08MB
+ predicates: c.c_custkey <= 2345, c.c_custkey >= 1234
+ row-size=218B cardinality=1.11K
+====
+# Multiple between predicates.
+select *
+from tpch.customer c
+where c.c_custkey between 1234 and 2345
+ and c.c_custkey between 1233 and 3446
+---- PLAN
+PLAN-ROOT SINK
+|
+00:SCAN HDFS [tpch.customer c]
+ HDFS partitions=1/1 files=1 size=23.08MB
+ predicates: c.c_custkey <= 2345, c.c_custkey <= 3446, c.c_custkey >= 1233, c.c_custkey >= 1234
+ row-size=218B cardinality=1.11K
+====
+# Redundant between predicates, where one of the bounding value overlap.
+select *
+from tpch.customer c
+where c.c_custkey between 1234 and 2345
+ and c.c_custkey between 1234 and 3456
+---- PLAN
+PLAN-ROOT SINK
+|
+00:SCAN HDFS [tpch.customer c]
+ HDFS partitions=1/1 files=1 size=23.08MB
+ predicates: c.c_custkey <= 2345, c.c_custkey <= 3456, c.c_custkey >= 1234
+ row-size=218B cardinality=1.11K
+====
+# Redundant between predicates, where both of the bounding value overlap.
+select *
+from tpch.customer c
+where c.c_custkey between 1234 and 2345
+ and c.c_custkey between 1234 and 2345
+---- PLAN
+PLAN-ROOT SINK
+|
+00:SCAN HDFS [tpch.customer c]
+ HDFS partitions=1/1 files=1 size=23.08MB
+ predicates: c.c_custkey <= 2345, c.c_custkey >= 1234
+ row-size=218B cardinality=1.11K
+====
+# Between and not between predicates that negate each other.
+# Bug: expected cardinality 0
+select *
+from tpch.customer c
+where c.c_custkey between 1234 and 2345
+ and c.c_custkey not between 1234 and 2345
+---- PLAN
+PLAN-ROOT SINK
+|
+00:SCAN HDFS [tpch.customer c]
+ HDFS partitions=1/1 files=1 size=23.08MB
+ predicates: c.c_custkey <= 2345, c.c_custkey >= 1234, c.c_custkey < 1234 OR c.c_custkey > 2345
+ row-size=218B cardinality=352
+====
+# Between and not between predicates only yield 2 valid values.
+# Bug: expected cardinality 2
+select *
+from tpch.customer c
+where c.c_custkey between 1234 and 2345
+ and c.c_custkey not between 1235 and 2344
+---- PLAN
+PLAN-ROOT SINK
+|
+00:SCAN HDFS [tpch.customer c]
+ HDFS partitions=1/1 files=1 size=23.08MB
+ predicates: c.c_custkey <= 2345, c.c_custkey >= 1234, c.c_custkey < 1235 OR c.c_custkey > 2344
+ row-size=218B cardinality=352
+====
+# Between and not between predicates where the latter has lower selectivity.
+select *
+from tpch.customer c
+where c.c_custkey between 1234 and 2345
+ and c.c_custkey not between 1000 and 150000
+---- PLAN
+PLAN-ROOT SINK
+|
+00:SCAN HDFS [tpch.customer c]
+ HDFS partitions=1/1 files=1 size=23.08MB
+ predicates: c.c_custkey <= 2345, c.c_custkey >= 1234, c.c_custkey < 1000 OR c.c_custkey > 150000
+ row-size=218B cardinality=999
+====
+# Between and not between predicates where the former has lower selectivity.
+select *
+from tpch.customer c
+where c.c_custkey between 1234 and 2345
+ and c.c_custkey not between 2000 and 150000
+---- PLAN
+PLAN-ROOT SINK
+|
+00:SCAN HDFS [tpch.customer c]
+ HDFS partitions=1/1 files=1 size=23.08MB
+ predicates: c.c_custkey <= 2345, c.c_custkey >= 1234, c.c_custkey < 2000 OR c.c_custkey > 150000
+ row-size=218B cardinality=1.11K
+====
+# BETWEEN should have low selectivity on unique INT column with many nulls.
+select * from functional_parquet.unique_with_nulls
+where int_col between 50 and 52
+---- PLAN
+PLAN-ROOT SINK
+|
+00:SCAN HDFS [functional_parquet.unique_with_nulls]
+ HDFS partitions=1/1 files=3 size=3.49KB
+ predicates: int_col <= 52, int_col >= 50
+ row-size=12B cardinality=3
+====
+# BETWEEN should have low selectivity on unique DATE column with many nulls.
+select * from functional_parquet.unique_with_nulls
+where date_col between DATE '2024-02-19' and DATE '2024-02-21'
+---- PLAN
+PLAN-ROOT SINK
+|
+00:SCAN HDFS [functional_parquet.unique_with_nulls]
+ HDFS partitions=1/1 files=3 size=3.49KB
+ predicates: date_col <= DATE '2024-02-21', date_col >= DATE '2024-02-19'
+ row-size=12B cardinality=3
====
# Partitioned scan.
# |table| = 11K
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q05.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q05.test
index 37462497b..a4ae67852 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q05.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q05.test
@@ -132,7 +132,7 @@ Max Per-Host Resource Reservation: Memory=75.56MB Threads=1
Per-Host Resource Estimates: Memory=10.95GB
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=10.95GB mem-reservation=75.56MB thread-reservation=1 runtime-filters-memory=23.00MB
-| max-parallelism=1 segment-costs=[16405209693, 7301848870, 5355066807, 342336, 221854, 313895, 500]
+| max-parallelism=1 segment-costs=[5637146480, 2816221067, 2740469898, 342336, 221854, 313895, 500]
PLAN-ROOT SINK
| output exprs: CASE valid_tid(27,28,29) WHEN 27 THEN channel WHEN 28 THEN channel WHEN 29 THEN NULL END, CASE valid_tid(27,28,29) WHEN 27 THEN id WHEN 28 THEN NULL WHEN 29 THEN NULL END, aggif(valid_tid(27,28,29) IN (27, 28, 29), CASE valid_tid(27,28,29) WHEN 27 THEN sum(sales) WHEN 28 THEN sum(sales) WHEN 29 THEN sum(sales) END), aggif(valid_tid(27,28,29) IN (27, 28, 29), CASE valid_tid(27,28,29) WHEN 27 THEN sum(`returns`) WHEN 28 THEN sum(`returns`) WHEN 29 THEN sum(`returns`) END), [...]
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=500
@@ -173,15 +173,33 @@ PLAN-ROOT SINK
| | output: sum(sales_price), sum(profit), sum(return_amt), sum(net_loss)
| | group by: web_site_id
| | mem-estimate=10.00MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=23 row-size=92B cardinality=33 cost=1176105559
+| | tuple-ids=23 row-size=92B cardinality=33 cost=4479919
| | in pipelines: 26(GETNEXT), 18(OPEN), 20(OPEN)
| |
| 25:HASH JOIN [INNER JOIN]
+| | hash predicates: wsr_web_site_sk = web_site_sk
+| | fk/pk conjuncts: assumed fk/pk
+| | runtime filters: RF008[bloom] <- web_site_sk, RF009[min_max] <- web_site_sk
+| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=19,21,22 row-size=64B cardinality=8.92M cost=3936848
+| | in pipelines: 18(GETNEXT), 20(GETNEXT), 23(OPEN)
+| |
+| |--23:SCAN HDFS [tpcds_partitioned_parquet_snap.web_site]
+| | HDFS partitions=1/1 files=1 size=11.78KB
+| | stored statistics:
+| | table: rows=66 size=17.88KB
+| | columns: all
+| | extrapolated-rows=disabled max-scan-range-rows=66
+| | mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=0
+| | tuple-ids=22 row-size=32B cardinality=66 cost=19
+| | in pipelines: 23(GETNEXT)
+| |
+| 24:HASH JOIN [INNER JOIN]
| | hash predicates: date_sk = d_date_sk
| | fk/pk conjuncts: assumed fk/pk
-| | runtime filters: RF008[bloom] <- d_date_sk
+| | runtime filters: RF010[bloom] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=19,22,21 row-size=64B cardinality=2.34G cost=1024447833
+| | tuple-ids=19,21 row-size=32B cardinality=9.05M cost=611076377
| | in pipelines: 18(GETNEXT), 20(GETNEXT), 22(OPEN)
| |
| |--22:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
@@ -194,27 +212,9 @@ PLAN-ROOT SINK
| | parquet statistics predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
| | parquet dictionary predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=21 row-size=8B cardinality=7.30K cost=12520
+| | tuple-ids=21 row-size=8B cardinality=15 cost=12520
| | in pipelines: 22(GETNEXT)
| |
-| 24:HASH JOIN [INNER JOIN]
-| | hash predicates: wsr_web_site_sk = web_site_sk
-| | fk/pk conjuncts: assumed fk/pk
-| | runtime filters: RF010[bloom] <- web_site_sk, RF011[min_max] <- web_site_sk
-| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=19,22 row-size=56B cardinality=2.34G cost=1033536661
-| | in pipelines: 18(GETNEXT), 20(GETNEXT), 23(OPEN)
-| |
-| |--23:SCAN HDFS [tpcds_partitioned_parquet_snap.web_site]
-| | HDFS partitions=1/1 files=1 size=11.78KB
-| | stored statistics:
-| | table: rows=66 size=17.88KB
-| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=66
-| | mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=0
-| | tuple-ids=22 row-size=32B cardinality=66 cost=19
-| | in pipelines: 23(GETNEXT)
-| |
| 17:UNION
| | mem-estimate=0B mem-reservation=0B thread-reservation=0
| | tuple-ids=19 row-size=24B cardinality=2.38G cost=241826293
@@ -230,7 +230,7 @@ PLAN-ROOT SINK
| | |
| | |--19:SCAN HDFS [tpcds_partitioned_parquet_snap.web_returns]
| | | HDFS partitions=2114/2114 files=2114 size=20.70MB
-| | | runtime filters: RF008[bloom] -> tpcds_partitioned_parquet_snap.web_returns.wr_returned_date_sk
+| | | runtime filters: RF010[bloom] -> tpcds_partitioned_parquet_snap.web_returns.wr_returned_date_sk
| | | stored statistics:
| | | table: rows=216.00M size=16.74GB
| | | partitions: 2114/2114 rows=216.00M
@@ -242,7 +242,7 @@ PLAN-ROOT SINK
| | |
| | 20:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales]
| | HDFS partitions=1824/1824 files=1824 size=84.18MB
-| | runtime filters: RF014[min_max] -> ws_item_sk, RF015[min_max] -> ws_order_number, RF011[min_max] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk, RF012[bloom] -> ws_item_sk, RF013[bloom] -> ws_order_number, RF010[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk
+| | runtime filters: RF014[min_max] -> ws_item_sk, RF015[min_max] -> ws_order_number, RF009[min_max] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk, RF012[bloom] -> ws_item_sk, RF013[bloom] -> ws_order_number, RF008[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk
| | stored statistics:
| | table: rows=2.16G size=145.75GB
| | partitions: 1824/1824 rows=2.16G
@@ -254,7 +254,7 @@ PLAN-ROOT SINK
| |
| 18:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales]
| HDFS partitions=1824/1824 files=1824 size=84.18MB
-| runtime filters: RF011[min_max] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk, RF010[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk, RF008[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_sold_date_sk
+| runtime filters: RF009[min_max] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk, RF010[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_sold_date_sk, RF008[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk
| stored statistics:
| table: rows=2.16G size=145.75GB
| partitions: 1824/1824 rows=2.16G
@@ -268,15 +268,33 @@ PLAN-ROOT SINK
| | output: sum(sales_price), sum(profit), sum(return_amt), sum(net_loss)
| | group by: cp_catalog_page_id
| | mem-estimate=10.00MB mem-reservation=4.75MB spill-buffer=256.00KB thread-reservation=0
-| | tuple-ids=14 row-size=92B cardinality=36.00K cost=2018439980
+| | tuple-ids=14 row-size=92B cardinality=36.00K cost=7874684
| | in pipelines: 16(GETNEXT), 10(OPEN), 11(OPEN)
| |
| 15:HASH JOIN [INNER JOIN]
+| | hash predicates: page_sk = cp_catalog_page_sk
+| | fk/pk conjuncts: assumed fk/pk
+| | runtime filters: RF004[bloom] <- cp_catalog_page_sk, RF005[min_max] <- cp_catalog_page_sk
+| | mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
+| | tuple-ids=10,12,13 row-size=64B cardinality=15.48M cost=7540710
+| | in pipelines: 10(GETNEXT), 11(GETNEXT), 13(OPEN)
+| |
+| |--13:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_page]
+| | HDFS partitions=1/1 files=1 size=739.17KB
+| | stored statistics:
+| | table: rows=36.00K size=2.24MB
+| | columns: all
+| | extrapolated-rows=disabled max-scan-range-rows=36.00K
+| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | tuple-ids=13 row-size=32B cardinality=36.00K cost=10368
+| | in pipelines: 13(GETNEXT)
+| |
+| 14:HASH JOIN [INNER JOIN]
| | hash predicates: date_sk = d_date_sk
| | fk/pk conjuncts: assumed fk/pk
-| | runtime filters: RF004[bloom] <- d_date_sk
+| | runtime filters: RF006[bloom] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=10,13,12 row-size=64B cardinality=4.02G cost=1758077150
+| | tuple-ids=10,12 row-size=32B cardinality=18.32M cost=1222231879
| | in pipelines: 10(GETNEXT), 11(GETNEXT), 12(OPEN)
| |
| |--12:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
@@ -289,27 +307,9 @@ PLAN-ROOT SINK
| | parquet statistics predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
| | parquet dictionary predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=12 row-size=8B cardinality=7.30K cost=12520
+| | tuple-ids=12 row-size=8B cardinality=15 cost=12520
| | in pipelines: 12(GETNEXT)
| |
-| 14:HASH JOIN [INNER JOIN]
-| | hash predicates: page_sk = cp_catalog_page_sk
-| | fk/pk conjuncts: assumed fk/pk
-| | runtime filters: RF006[bloom] <- cp_catalog_page_sk, RF007[min_max] <- cp_catalog_page_sk
-| | mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
-| | tuple-ids=10,13 row-size=56B cardinality=4.02G cost=1946757946
-| | in pipelines: 10(GETNEXT), 11(GETNEXT), 13(OPEN)
-| |
-| |--13:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_page]
-| | HDFS partitions=1/1 files=1 size=739.17KB
-| | stored statistics:
-| | table: rows=36.00K size=2.24MB
-| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=36.00K
-| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=13 row-size=32B cardinality=36.00K cost=10368
-| | in pipelines: 13(GETNEXT)
-| |
| 09:UNION
| | mem-estimate=0B mem-reservation=0B thread-reservation=0
| | tuple-ids=10 row-size=24B cardinality=4.75G cost=483668226
@@ -317,7 +317,7 @@ PLAN-ROOT SINK
| |
| |--11:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_returns]
| | HDFS partitions=2060/2060 files=2060 size=30.47MB
-| | runtime filters: RF007[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_catalog_page_sk, RF006[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_catalog_page_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_returned_date_sk
+| | runtime filters: RF005[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_catalog_page_sk, RF006[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_returned_date_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_catalog_page_sk
| | stored statistics:
| | table: rows=432.02M size=32.77GB
| | partitions: 2060/2060 rows=432.02M
@@ -329,7 +329,7 @@ PLAN-ROOT SINK
| |
| 10:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales]
| HDFS partitions=1831/1831 files=1831 size=150.82MB
-| runtime filters: RF007[min_max] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_catalog_page_sk, RF006[bloom] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_catalog_page_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_sold_date_sk
+| runtime filters: RF005[min_max] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_catalog_page_sk, RF006[bloom] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_sold_date_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_catalog_page_sk
| stored statistics:
| table: rows=4.32G size=280.96GB
| partitions: 1831/1831 rows=4.32G
@@ -343,7 +343,7 @@ PLAN-ROOT SINK
| output: sum(sales_price), sum(profit), sum(return_amt), sum(net_loss)
| group by: s_store_id
| mem-estimate=10.00MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=6 row-size=92B cardinality=678 cost=4887965622
+| tuple-ids=6 row-size=92B cardinality=678 cost=19155231
| in pipelines: 08(GETNEXT), 02(OPEN), 03(OPEN)
|
07:HASH JOIN [INNER JOIN]
@@ -351,7 +351,7 @@ PLAN-ROOT SINK
| fk/pk conjuncts: assumed fk/pk
| runtime filters: RF000[bloom] <- s_store_sk, RF001[min_max] <- s_store_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=2,4,5 row-size=64B cardinality=9.73G cost=4200342986
+| tuple-ids=2,4,5 row-size=64B cardinality=38.12M cost=16460368
| in pipelines: 02(GETNEXT), 03(GETNEXT), 05(OPEN)
|
|--05:SCAN HDFS [tpcds_partitioned_parquet_snap.store]
@@ -369,7 +369,7 @@ PLAN-ROOT SINK
| fk/pk conjuncts: assumed fk/pk
| runtime filters: RF002[bloom] <- d_date_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=2,4 row-size=32B cardinality=9.50G cost=4159874763
+| tuple-ids=2,4 row-size=32B cardinality=37.24M cost=2444504559
| in pipelines: 02(GETNEXT), 03(GETNEXT), 04(OPEN)
|
|--04:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
@@ -382,7 +382,7 @@ PLAN-ROOT SINK
| parquet statistics predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
| parquet dictionary predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=4 row-size=8B cardinality=7.30K cost=12520
+| tuple-ids=4 row-size=8B cardinality=15 cost=12520
| in pipelines: 04(GETNEXT)
|
01:UNION
@@ -502,86 +502,86 @@ max-parallelism=120 segment-costs=[2267, 130557, 33, 344492, 567467] cpu-compari
| F16:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
| Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
| Per-Instance Resources: mem-estimate=71.00MB mem-reservation=2.12MB thread-reservation=1
-| max-parallelism=120 segment-costs=[4965693385, 212] cpu-comparison-result=240 [max(240 (self) vs 202 (sum children))]
+| max-parallelism=120 segment-costs=[2351103766, 212] cpu-comparison-result=240 [max(240 (self) vs 202 (sum children))]
| 26:AGGREGATE [STREAMING]
| | output: sum(sales_price), sum(profit), sum(return_amt), sum(net_loss)
| | group by: web_site_id
| | mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=23 row-size=92B cardinality=33 cost=1176115797
+| | tuple-ids=23 row-size=92B cardinality=33 cost=4490157
| | in pipelines: 18(GETNEXT), 20(GETNEXT)
| |
| 25:HASH JOIN [INNER JOIN, BROADCAST]
| | hash-table-id=04
-| | hash predicates: date_sk = d_date_sk
+| | hash predicates: wsr_web_site_sk = web_site_sk
| | fk/pk conjuncts: assumed fk/pk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=19,22,21 row-size=64B cardinality=2.34G cost=1024440528
-| | in pipelines: 18(GETNEXT), 20(GETNEXT), 22(OPEN)
+| | tuple-ids=19,21,22 row-size=64B cardinality=8.92M cost=3936782
+| | in pipelines: 18(GETNEXT), 20(GETNEXT), 23(OPEN)
| |
| |--F27:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[17005]
+| | | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[146]
| | JOIN BUILD
| | | join-table-id=04 plan-id=05 cohort-id=03
-| | | build expressions: d_date_sk
-| | | runtime filters: RF008[bloom] <- d_date_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | | build expressions: web_site_sk
+| | | runtime filters: RF008[bloom] <- web_site_sk, RF009[min_max] <- web_site_sk
+| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=66
| | |
| | 41:EXCHANGE [BROADCAST]
-| | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=21 row-size=8B cardinality=7.30K cost=9700
-| | | in pipelines: 22(GETNEXT)
+| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=22 row-size=32B cardinality=66 cost=80
+| | | in pipelines: 23(GETNEXT)
| | |
| | F18:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[12835]
-| | 22:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| | HDFS partitions=1/1 files=1 size=2.17MB
-| | predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
+| | Per-Instance Resources: mem-estimate=16.14MB mem-reservation=16.00KB thread-reservation=1
+| | max-parallelism=1 segment-costs=[26]
+| | 23:SCAN HDFS [tpcds_partitioned_parquet_snap.web_site, RANDOM]
+| | HDFS partitions=1/1 files=1 size=11.78KB
| | stored statistics:
-| | table: rows=73.05K size=2.17MB
+| | table: rows=66 size=17.88KB
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=73.05K
-| | parquet statistics predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
-| | parquet dictionary predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
-| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=21 row-size=8B cardinality=7.30K cost=12520
-| | in pipelines: 22(GETNEXT)
+| | extrapolated-rows=disabled max-scan-range-rows=66
+| | mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=0
+| | tuple-ids=22 row-size=32B cardinality=66 cost=19
+| | in pipelines: 23(GETNEXT)
| |
| 24:HASH JOIN [INNER JOIN, BROADCAST]
| | hash-table-id=05
-| | hash predicates: wsr_web_site_sk = web_site_sk
+| | hash predicates: date_sk = d_date_sk
| | fk/pk conjuncts: assumed fk/pk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=19,22 row-size=56B cardinality=2.34G cost=1033536595
-| | in pipelines: 18(GETNEXT), 20(GETNEXT), 23(OPEN)
+| | tuple-ids=19,21 row-size=32B cardinality=9.05M cost=611076362
+| | in pipelines: 18(GETNEXT), 20(GETNEXT), 22(OPEN)
| |
| |--F28:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
| | | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[146]
+| | | max-parallelism=10 segment-costs=[25]
| | JOIN BUILD
| | | join-table-id=05 plan-id=06 cohort-id=03
-| | | build expressions: web_site_sk
-| | | runtime filters: RF010[bloom] <- web_site_sk, RF011[min_max] <- web_site_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=66
+| | | build expressions: d_date_sk
+| | | runtime filters: RF010[bloom] <- d_date_sk
+| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=15
| | |
| | 40:EXCHANGE [BROADCAST]
| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=22 row-size=32B cardinality=66 cost=80
-| | | in pipelines: 23(GETNEXT)
+| | | tuple-ids=21 row-size=8B cardinality=15 cost=10
+| | | in pipelines: 22(GETNEXT)
| | |
| | F17:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | Per-Instance Resources: mem-estimate=16.14MB mem-reservation=16.00KB thread-reservation=1
-| | max-parallelism=1 segment-costs=[26]
-| | 23:SCAN HDFS [tpcds_partitioned_parquet_snap.web_site, RANDOM]
-| | HDFS partitions=1/1 files=1 size=11.78KB
+| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| | max-parallelism=1 segment-costs=[12520]
+| | 22:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| | HDFS partitions=1/1 files=1 size=2.17MB
+| | predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
| | stored statistics:
-| | table: rows=66 size=17.88KB
+| | table: rows=73.05K size=2.17MB
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=66
-| | mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=0
-| | tuple-ids=22 row-size=32B cardinality=66 cost=19
-| | in pipelines: 23(GETNEXT)
+| | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | parquet statistics predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
+| | parquet dictionary predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
+| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | tuple-ids=21 row-size=8B cardinality=15 cost=12520
+| | in pipelines: 22(GETNEXT)
| |
| 17:UNION
| | mem-estimate=0B mem-reservation=0B thread-reservation=0
@@ -616,7 +616,7 @@ max-parallelism=120 segment-costs=[2267, 130557, 33, 344492, 567467] cpu-compari
| | | max-parallelism=60 segment-costs=[519788863]
| | | 19:SCAN HDFS [tpcds_partitioned_parquet_snap.web_returns, RANDOM]
| | | HDFS partitions=2114/2114 files=2114 size=20.70MB
-| | | runtime filters: RF008[bloom] -> tpcds_partitioned_parquet_snap.web_returns.wr_returned_date_sk
+| | | runtime filters: RF010[bloom] -> tpcds_partitioned_parquet_snap.web_returns.wr_returned_date_sk
| | | stored statistics:
| | | table: rows=216.00M size=16.74GB
| | | partitions: 2114/2114 rows=216.00M
@@ -637,7 +637,7 @@ max-parallelism=120 segment-costs=[2267, 130557, 33, 344492, 567467] cpu-compari
| | max-parallelism=390 segment-costs=[3836103108]
| | 20:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales, RANDOM]
| | HDFS partitions=1824/1824 files=1824 size=84.18MB
-| | runtime filters: RF014[min_max] -> ws_item_sk, RF015[min_max] -> ws_order_number, RF011[min_max] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk, RF012[bloom] -> ws_item_sk, RF013[bloom] -> ws_order_number, RF010[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk
+| | runtime filters: RF014[min_max] -> ws_item_sk, RF015[min_max] -> ws_order_number, RF009[min_max] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk, RF012[bloom] -> ws_item_sk, RF013[bloom] -> ws_order_number, RF008[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk
| | stored statistics:
| | table: rows=2.16G size=145.75GB
| | partitions: 1824/1824 rows=2.16G
@@ -649,7 +649,7 @@ max-parallelism=120 segment-costs=[2267, 130557, 33, 344492, 567467] cpu-compari
| |
| 18:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales, RANDOM]
| HDFS partitions=1824/1824 files=1824 size=84.18MB
-| runtime filters: RF011[min_max] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk, RF010[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk, RF008[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_sold_date_sk
+| runtime filters: RF009[min_max] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk, RF010[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_sold_date_sk, RF008[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk
| stored statistics:
| table: rows=2.16G size=145.75GB
| partitions: 1824/1824 rows=2.16G
@@ -674,86 +674,86 @@ max-parallelism=120 segment-costs=[2267, 130557, 33, 344492, 567467] cpu-compari
| F08:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
| Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
| Per-Instance Resources: mem-estimate=71.00MB mem-reservation=6.00MB thread-reservation=1
-| max-parallelism=120 segment-costs=[7312951922, 231840] cpu-comparison-result=120 [max(120 (self) vs 22 (sum children))]
+| max-parallelism=120 segment-costs=[2827018545, 231840] cpu-comparison-result=120 [max(120 (self) vs 22 (sum children))]
| 16:AGGREGATE [STREAMING]
| | output: sum(sales_price), sum(profit), sum(return_amt), sum(net_loss)
| | group by: cp_catalog_page_id
| | mem-estimate=10.00MB mem-reservation=5.00MB spill-buffer=256.00KB thread-reservation=0
-| | tuple-ids=14 row-size=92B cardinality=36.00K cost=2029609225
+| | tuple-ids=14 row-size=92B cardinality=36.00K cost=18731065
| | in pipelines: 10(GETNEXT), 11(GETNEXT)
| |
| 15:HASH JOIN [INNER JOIN, BROADCAST]
| | hash-table-id=02
-| | hash predicates: date_sk = d_date_sk
+| | hash predicates: page_sk = cp_catalog_page_sk
| | fk/pk conjuncts: assumed fk/pk
-| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=10,13,12 row-size=64B cardinality=4.02G cost=1758069845
-| | in pipelines: 10(GETNEXT), 11(GETNEXT), 12(OPEN)
+| | mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
+| | tuple-ids=10,12,13 row-size=64B cardinality=15.48M cost=7504710
+| | in pipelines: 10(GETNEXT), 11(GETNEXT), 13(OPEN)
| |
| |--F25:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[17005]
+| | | Per-Instance Resources: mem-estimate=36.63MB mem-reservation=35.50MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[83840]
| | JOIN BUILD
| | | join-table-id=02 plan-id=03 cohort-id=02
-| | | build expressions: d_date_sk
-| | | runtime filters: RF004[bloom] <- d_date_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | | build expressions: cp_catalog_page_sk
+| | | runtime filters: RF004[bloom] <- cp_catalog_page_sk, RF005[min_max] <- cp_catalog_page_sk
+| | | mem-estimate=34.50MB mem-reservation=34.50MB spill-buffer=128.00KB thread-reservation=0 cost=36000
| | |
| | 35:EXCHANGE [BROADCAST]
-| | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=12 row-size=8B cardinality=7.30K cost=9700
-| | | in pipelines: 12(GETNEXT)
+| | | mem-estimate=1.13MB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=13 row-size=32B cardinality=36.00K cost=47840
+| | | in pipelines: 13(GETNEXT)
| | |
| | F10:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[12835]
-| | 12:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| | HDFS partitions=1/1 files=1 size=2.17MB
-| | predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
+| | Per-Instance Resources: mem-estimate=16.14MB mem-reservation=1.00MB thread-reservation=1
+| | max-parallelism=1 segment-costs=[14256]
+| | 13:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_page, RANDOM]
+| | HDFS partitions=1/1 files=1 size=739.17KB
| | stored statistics:
-| | table: rows=73.05K size=2.17MB
+| | table: rows=36.00K size=2.24MB
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=73.05K
-| | parquet statistics predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
-| | parquet dictionary predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
+| | extrapolated-rows=disabled max-scan-range-rows=36.00K
| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=12 row-size=8B cardinality=7.30K cost=12520
-| | in pipelines: 12(GETNEXT)
+| | tuple-ids=13 row-size=32B cardinality=36.00K cost=10368
+| | in pipelines: 13(GETNEXT)
| |
| 14:HASH JOIN [INNER JOIN, BROADCAST]
| | hash-table-id=03
-| | hash predicates: page_sk = cp_catalog_page_sk
+| | hash predicates: date_sk = d_date_sk
| | fk/pk conjuncts: assumed fk/pk
-| | mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
-| | tuple-ids=10,13 row-size=56B cardinality=4.02G cost=1946721946
-| | in pipelines: 10(GETNEXT), 11(GETNEXT), 13(OPEN)
+| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=10,12 row-size=32B cardinality=18.32M cost=1222231864
+| | in pipelines: 10(GETNEXT), 11(GETNEXT), 12(OPEN)
| |
| |--F26:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=36.63MB mem-reservation=35.50MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[83840]
+| | | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[25]
| | JOIN BUILD
| | | join-table-id=03 plan-id=04 cohort-id=02
-| | | build expressions: cp_catalog_page_sk
-| | | runtime filters: RF006[bloom] <- cp_catalog_page_sk, RF007[min_max] <- cp_catalog_page_sk
-| | | mem-estimate=34.50MB mem-reservation=34.50MB spill-buffer=128.00KB thread-reservation=0 cost=36000
+| | | build expressions: d_date_sk
+| | | runtime filters: RF006[bloom] <- d_date_sk
+| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=15
| | |
| | 34:EXCHANGE [BROADCAST]
-| | | mem-estimate=1.13MB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=13 row-size=32B cardinality=36.00K cost=47840
-| | | in pipelines: 13(GETNEXT)
+| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=12 row-size=8B cardinality=15 cost=10
+| | | in pipelines: 12(GETNEXT)
| | |
| | F09:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | Per-Instance Resources: mem-estimate=16.14MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[14256]
-| | 13:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_page, RANDOM]
-| | HDFS partitions=1/1 files=1 size=739.17KB
+| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| | max-parallelism=1 segment-costs=[12520]
+| | 12:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| | HDFS partitions=1/1 files=1 size=2.17MB
+| | predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
| | stored statistics:
-| | table: rows=36.00K size=2.24MB
+| | table: rows=73.05K size=2.17MB
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=36.00K
+| | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | parquet statistics predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
+| | parquet dictionary predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=13 row-size=32B cardinality=36.00K cost=10368
-| | in pipelines: 13(GETNEXT)
+| | tuple-ids=12 row-size=8B cardinality=15 cost=12520
+| | in pipelines: 12(GETNEXT)
| |
| 09:UNION
| | mem-estimate=0B mem-reservation=0B thread-reservation=0
@@ -762,7 +762,7 @@ max-parallelism=120 segment-costs=[2267, 130557, 33, 344492, 567467] cpu-compari
| |
| |--11:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_returns, RANDOM]
| | HDFS partitions=2060/2060 files=2060 size=30.47MB
-| | runtime filters: RF007[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_catalog_page_sk, RF006[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_catalog_page_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_returned_date_sk
+| | runtime filters: RF005[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_catalog_page_sk, RF006[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_returned_date_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_catalog_page_sk
| | stored statistics:
| | table: rows=432.02M size=32.77GB
| | partitions: 2060/2060 rows=432.02M
@@ -774,7 +774,7 @@ max-parallelism=120 segment-costs=[2267, 130557, 33, 344492, 567467] cpu-compari
| |
| 10:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales, RANDOM]
| HDFS partitions=1831/1831 files=1831 size=150.82MB
-| runtime filters: RF007[min_max] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_catalog_page_sk, RF006[bloom] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_catalog_page_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_sold_date_sk
+| runtime filters: RF005[min_max] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_catalog_page_sk, RF006[bloom] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_sold_date_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_catalog_page_sk
| stored statistics:
| table: rows=4.32G size=280.96GB
| partitions: 1831/1831 rows=4.32G
@@ -799,12 +799,12 @@ max-parallelism=120 segment-costs=[2267, 130557, 33, 344492, 567467] cpu-compari
F02:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
Per-Instance Resources: mem-estimate=71.00MB mem-reservation=10.00MB thread-reservation=1
-max-parallelism=120 segment-costs=[16405398484, 4366] cpu-comparison-result=120 [max(120 (self) vs 22 (sum children))]
+max-parallelism=120 segment-costs=[5637342561, 4366] cpu-comparison-result=120 [max(120 (self) vs 22 (sum children))]
08:AGGREGATE [STREAMING]
| output: sum(sales_price), sum(profit), sum(return_amt), sum(net_loss)
| group by: s_store_id
| mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=6 row-size=92B cardinality=678 cost=4888175976
+| tuple-ids=6 row-size=92B cardinality=678 cost=19365585
| in pipelines: 02(GETNEXT), 03(GETNEXT)
|
07:HASH JOIN [INNER JOIN, BROADCAST]
@@ -812,7 +812,7 @@ max-parallelism=120 segment-costs=[16405398484, 4366] cpu-comparison-result=120
| hash predicates: store_sk = s_store_sk
| fk/pk conjuncts: assumed fk/pk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=2,4,5 row-size=64B cardinality=9.73G cost=4200341636
+| tuple-ids=2,4,5 row-size=64B cardinality=38.12M cost=16459018
| in pipelines: 02(GETNEXT), 03(GETNEXT), 05(OPEN)
|
|--F23:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
@@ -847,26 +847,26 @@ max-parallelism=120 segment-costs=[16405398484, 4366] cpu-comparison-result=120
| hash predicates: date_sk = d_date_sk
| fk/pk conjuncts: assumed fk/pk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=2,4 row-size=32B cardinality=9.50G cost=4159867458
+| tuple-ids=2,4 row-size=32B cardinality=37.24M cost=2444504544
| in pipelines: 02(GETNEXT), 03(GETNEXT), 04(OPEN)
|
|--F24:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
+| | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[25]
| JOIN BUILD
| | join-table-id=01 plan-id=02 cohort-id=01
| | build expressions: d_date_sk
| | runtime filters: RF002[bloom] <- d_date_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=15
| |
| 30:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=4 row-size=8B cardinality=7.30K cost=9700
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=4 row-size=8B cardinality=15 cost=10
| | in pipelines: 04(GETNEXT)
| |
| F03:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
+| max-parallelism=1 segment-costs=[12520]
| 04:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| HDFS partitions=1/1 files=1 size=2.17MB
| predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
@@ -877,7 +877,7 @@ max-parallelism=120 segment-costs=[16405398484, 4366] cpu-comparison-result=120
| parquet statistics predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
| parquet dictionary predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=4 row-size=8B cardinality=7.30K cost=12520
+| tuple-ids=4 row-size=8B cardinality=15 cost=12520
| in pipelines: 04(GETNEXT)
|
01:UNION
@@ -997,86 +997,86 @@ max-parallelism=120 segment-costs=[2267, 130557, 33, 344492, 567467] cpu-compari
| F16:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
| Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
| Per-Instance Resources: mem-estimate=71.00MB mem-reservation=2.12MB thread-reservation=1
-| max-parallelism=120 segment-costs=[4965693385, 212] cpu-comparison-result=240 [max(240 (self) vs 202 (sum children))]
+| max-parallelism=120 segment-costs=[2351103766, 212] cpu-comparison-result=240 [max(240 (self) vs 202 (sum children))]
| 26:AGGREGATE [STREAMING]
| | output: sum(sales_price), sum(profit), sum(return_amt), sum(net_loss)
| | group by: web_site_id
| | mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=23 row-size=92B cardinality=33 cost=1176115797
+| | tuple-ids=23 row-size=92B cardinality=33 cost=4490157
| | in pipelines: 18(GETNEXT), 20(GETNEXT)
| |
| 25:HASH JOIN [INNER JOIN, BROADCAST]
| | hash-table-id=04
-| | hash predicates: date_sk = d_date_sk
+| | hash predicates: wsr_web_site_sk = web_site_sk
| | fk/pk conjuncts: assumed fk/pk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=19,22,21 row-size=64B cardinality=2.34G cost=1024440528
-| | in pipelines: 18(GETNEXT), 20(GETNEXT), 22(OPEN)
+| | tuple-ids=19,21,22 row-size=64B cardinality=8.92M cost=3936782
+| | in pipelines: 18(GETNEXT), 20(GETNEXT), 23(OPEN)
| |
| |--F27:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[17005]
+| | | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[146]
| | JOIN BUILD
| | | join-table-id=04 plan-id=05 cohort-id=03
-| | | build expressions: d_date_sk
-| | | runtime filters: RF008[bloom] <- d_date_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | | build expressions: web_site_sk
+| | | runtime filters: RF008[bloom] <- web_site_sk, RF009[min_max] <- web_site_sk
+| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=66
| | |
| | 41:EXCHANGE [BROADCAST]
-| | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=21 row-size=8B cardinality=7.30K cost=9700
-| | | in pipelines: 22(GETNEXT)
+| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=22 row-size=32B cardinality=66 cost=80
+| | | in pipelines: 23(GETNEXT)
| | |
| | F18:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[12835]
-| | 22:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| | HDFS partitions=1/1 files=1 size=2.17MB
-| | predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
+| | Per-Instance Resources: mem-estimate=16.14MB mem-reservation=16.00KB thread-reservation=1
+| | max-parallelism=1 segment-costs=[26]
+| | 23:SCAN HDFS [tpcds_partitioned_parquet_snap.web_site, RANDOM]
+| | HDFS partitions=1/1 files=1 size=11.78KB
| | stored statistics:
-| | table: rows=73.05K size=2.17MB
+| | table: rows=66 size=17.88KB
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=73.05K
-| | parquet statistics predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
-| | parquet dictionary predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
-| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=21 row-size=8B cardinality=7.30K cost=12520
-| | in pipelines: 22(GETNEXT)
+| | extrapolated-rows=disabled max-scan-range-rows=66
+| | mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=0
+| | tuple-ids=22 row-size=32B cardinality=66 cost=19
+| | in pipelines: 23(GETNEXT)
| |
| 24:HASH JOIN [INNER JOIN, BROADCAST]
| | hash-table-id=05
-| | hash predicates: wsr_web_site_sk = web_site_sk
+| | hash predicates: date_sk = d_date_sk
| | fk/pk conjuncts: assumed fk/pk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=19,22 row-size=56B cardinality=2.34G cost=1033536595
-| | in pipelines: 18(GETNEXT), 20(GETNEXT), 23(OPEN)
+| | tuple-ids=19,21 row-size=32B cardinality=9.05M cost=611076362
+| | in pipelines: 18(GETNEXT), 20(GETNEXT), 22(OPEN)
| |
| |--F28:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
| | | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[146]
+| | | max-parallelism=10 segment-costs=[25]
| | JOIN BUILD
| | | join-table-id=05 plan-id=06 cohort-id=03
-| | | build expressions: web_site_sk
-| | | runtime filters: RF010[bloom] <- web_site_sk, RF011[min_max] <- web_site_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=66
+| | | build expressions: d_date_sk
+| | | runtime filters: RF010[bloom] <- d_date_sk
+| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=15
| | |
| | 40:EXCHANGE [BROADCAST]
| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=22 row-size=32B cardinality=66 cost=80
-| | | in pipelines: 23(GETNEXT)
+| | | tuple-ids=21 row-size=8B cardinality=15 cost=10
+| | | in pipelines: 22(GETNEXT)
| | |
| | F17:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | Per-Instance Resources: mem-estimate=16.14MB mem-reservation=16.00KB thread-reservation=1
-| | max-parallelism=1 segment-costs=[26]
-| | 23:SCAN HDFS [tpcds_partitioned_parquet_snap.web_site, RANDOM]
-| | HDFS partitions=1/1 files=1 size=11.78KB
+| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| | max-parallelism=1 segment-costs=[12520]
+| | 22:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| | HDFS partitions=1/1 files=1 size=2.17MB
+| | predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
| | stored statistics:
-| | table: rows=66 size=17.88KB
+| | table: rows=73.05K size=2.17MB
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=66
-| | mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=0
-| | tuple-ids=22 row-size=32B cardinality=66 cost=19
-| | in pipelines: 23(GETNEXT)
+| | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | parquet statistics predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
+| | parquet dictionary predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
+| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | tuple-ids=21 row-size=8B cardinality=15 cost=12520
+| | in pipelines: 22(GETNEXT)
| |
| 17:UNION
| | mem-estimate=0B mem-reservation=0B thread-reservation=0
@@ -1111,7 +1111,7 @@ max-parallelism=120 segment-costs=[2267, 130557, 33, 344492, 567467] cpu-compari
| | | max-parallelism=60 segment-costs=[519788863]
| | | 19:SCAN HDFS [tpcds_partitioned_parquet_snap.web_returns, RANDOM]
| | | HDFS partitions=2114/2114 files=2114 size=20.70MB
-| | | runtime filters: RF008[bloom] -> tpcds_partitioned_parquet_snap.web_returns.wr_returned_date_sk
+| | | runtime filters: RF010[bloom] -> tpcds_partitioned_parquet_snap.web_returns.wr_returned_date_sk
| | | stored statistics:
| | | table: rows=216.00M size=16.74GB
| | | partitions: 2114/2114 rows=216.00M
@@ -1132,7 +1132,7 @@ max-parallelism=120 segment-costs=[2267, 130557, 33, 344492, 567467] cpu-compari
| | max-parallelism=390 segment-costs=[3836103108]
| | 20:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales, RANDOM]
| | HDFS partitions=1824/1824 files=1824 size=84.18MB
-| | runtime filters: RF014[min_max] -> ws_item_sk, RF015[min_max] -> ws_order_number, RF011[min_max] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk, RF012[bloom] -> ws_item_sk, RF013[bloom] -> ws_order_number, RF010[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk
+| | runtime filters: RF014[min_max] -> ws_item_sk, RF015[min_max] -> ws_order_number, RF009[min_max] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk, RF012[bloom] -> ws_item_sk, RF013[bloom] -> ws_order_number, RF008[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk
| | stored statistics:
| | table: rows=2.16G size=145.75GB
| | partitions: 1824/1824 rows=2.16G
@@ -1144,7 +1144,7 @@ max-parallelism=120 segment-costs=[2267, 130557, 33, 344492, 567467] cpu-compari
| |
| 18:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales, RANDOM]
| HDFS partitions=1824/1824 files=1824 size=84.18MB
-| runtime filters: RF011[min_max] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk, RF010[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk, RF008[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_sold_date_sk
+| runtime filters: RF009[min_max] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk, RF010[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_sold_date_sk, RF008[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_web_site_sk
| stored statistics:
| table: rows=2.16G size=145.75GB
| partitions: 1824/1824 rows=2.16G
@@ -1169,86 +1169,86 @@ max-parallelism=120 segment-costs=[2267, 130557, 33, 344492, 567467] cpu-compari
| F08:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
| Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
| Per-Instance Resources: mem-estimate=71.00MB mem-reservation=6.00MB thread-reservation=1
-| max-parallelism=120 segment-costs=[7312951922, 231840] cpu-comparison-result=120 [max(120 (self) vs 22 (sum children))]
+| max-parallelism=120 segment-costs=[2827018545, 231840] cpu-comparison-result=120 [max(120 (self) vs 22 (sum children))]
| 16:AGGREGATE [STREAMING]
| | output: sum(sales_price), sum(profit), sum(return_amt), sum(net_loss)
| | group by: cp_catalog_page_id
| | mem-estimate=10.00MB mem-reservation=5.00MB spill-buffer=256.00KB thread-reservation=0
-| | tuple-ids=14 row-size=92B cardinality=36.00K cost=2029609225
+| | tuple-ids=14 row-size=92B cardinality=36.00K cost=18731065
| | in pipelines: 10(GETNEXT), 11(GETNEXT)
| |
| 15:HASH JOIN [INNER JOIN, BROADCAST]
| | hash-table-id=02
-| | hash predicates: date_sk = d_date_sk
+| | hash predicates: page_sk = cp_catalog_page_sk
| | fk/pk conjuncts: assumed fk/pk
-| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=10,13,12 row-size=64B cardinality=4.02G cost=1758069845
-| | in pipelines: 10(GETNEXT), 11(GETNEXT), 12(OPEN)
+| | mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
+| | tuple-ids=10,12,13 row-size=64B cardinality=15.48M cost=7504710
+| | in pipelines: 10(GETNEXT), 11(GETNEXT), 13(OPEN)
| |
| |--F25:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[17005]
+| | | Per-Instance Resources: mem-estimate=36.63MB mem-reservation=35.50MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[83840]
| | JOIN BUILD
| | | join-table-id=02 plan-id=03 cohort-id=02
-| | | build expressions: d_date_sk
-| | | runtime filters: RF004[bloom] <- d_date_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | | build expressions: cp_catalog_page_sk
+| | | runtime filters: RF004[bloom] <- cp_catalog_page_sk, RF005[min_max] <- cp_catalog_page_sk
+| | | mem-estimate=34.50MB mem-reservation=34.50MB spill-buffer=128.00KB thread-reservation=0 cost=36000
| | |
| | 35:EXCHANGE [BROADCAST]
-| | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=12 row-size=8B cardinality=7.30K cost=9700
-| | | in pipelines: 12(GETNEXT)
+| | | mem-estimate=1.13MB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=13 row-size=32B cardinality=36.00K cost=47840
+| | | in pipelines: 13(GETNEXT)
| | |
| | F10:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[12835]
-| | 12:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| | HDFS partitions=1/1 files=1 size=2.17MB
-| | predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
+| | Per-Instance Resources: mem-estimate=16.14MB mem-reservation=1.00MB thread-reservation=1
+| | max-parallelism=1 segment-costs=[14256]
+| | 13:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_page, RANDOM]
+| | HDFS partitions=1/1 files=1 size=739.17KB
| | stored statistics:
-| | table: rows=73.05K size=2.17MB
+| | table: rows=36.00K size=2.24MB
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=73.05K
-| | parquet statistics predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
-| | parquet dictionary predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
+| | extrapolated-rows=disabled max-scan-range-rows=36.00K
| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=12 row-size=8B cardinality=7.30K cost=12520
-| | in pipelines: 12(GETNEXT)
+| | tuple-ids=13 row-size=32B cardinality=36.00K cost=10368
+| | in pipelines: 13(GETNEXT)
| |
| 14:HASH JOIN [INNER JOIN, BROADCAST]
| | hash-table-id=03
-| | hash predicates: page_sk = cp_catalog_page_sk
+| | hash predicates: date_sk = d_date_sk
| | fk/pk conjuncts: assumed fk/pk
-| | mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
-| | tuple-ids=10,13 row-size=56B cardinality=4.02G cost=1946721946
-| | in pipelines: 10(GETNEXT), 11(GETNEXT), 13(OPEN)
+| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=10,12 row-size=32B cardinality=18.32M cost=1222231864
+| | in pipelines: 10(GETNEXT), 11(GETNEXT), 12(OPEN)
| |
| |--F26:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=36.63MB mem-reservation=35.50MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[83840]
+| | | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[25]
| | JOIN BUILD
| | | join-table-id=03 plan-id=04 cohort-id=02
-| | | build expressions: cp_catalog_page_sk
-| | | runtime filters: RF006[bloom] <- cp_catalog_page_sk, RF007[min_max] <- cp_catalog_page_sk
-| | | mem-estimate=34.50MB mem-reservation=34.50MB spill-buffer=128.00KB thread-reservation=0 cost=36000
+| | | build expressions: d_date_sk
+| | | runtime filters: RF006[bloom] <- d_date_sk
+| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=15
| | |
| | 34:EXCHANGE [BROADCAST]
-| | | mem-estimate=1.13MB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=13 row-size=32B cardinality=36.00K cost=47840
-| | | in pipelines: 13(GETNEXT)
+| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=12 row-size=8B cardinality=15 cost=10
+| | | in pipelines: 12(GETNEXT)
| | |
| | F09:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | Per-Instance Resources: mem-estimate=16.14MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[14256]
-| | 13:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_page, RANDOM]
-| | HDFS partitions=1/1 files=1 size=739.17KB
+| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| | max-parallelism=1 segment-costs=[12520]
+| | 12:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| | HDFS partitions=1/1 files=1 size=2.17MB
+| | predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
| | stored statistics:
-| | table: rows=36.00K size=2.24MB
+| | table: rows=73.05K size=2.17MB
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=36.00K
+| | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | parquet statistics predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
+| | parquet dictionary predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=13 row-size=32B cardinality=36.00K cost=10368
-| | in pipelines: 13(GETNEXT)
+| | tuple-ids=12 row-size=8B cardinality=15 cost=12520
+| | in pipelines: 12(GETNEXT)
| |
| 09:UNION
| | mem-estimate=0B mem-reservation=0B thread-reservation=0
@@ -1257,7 +1257,7 @@ max-parallelism=120 segment-costs=[2267, 130557, 33, 344492, 567467] cpu-compari
| |
| |--11:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_returns, RANDOM]
| | HDFS partitions=2060/2060 files=2060 size=30.47MB
-| | runtime filters: RF007[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_catalog_page_sk, RF006[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_catalog_page_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_returned_date_sk
+| | runtime filters: RF005[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_catalog_page_sk, RF006[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_returned_date_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_catalog_page_sk
| | stored statistics:
| | table: rows=432.02M size=32.77GB
| | partitions: 2060/2060 rows=432.02M
@@ -1269,7 +1269,7 @@ max-parallelism=120 segment-costs=[2267, 130557, 33, 344492, 567467] cpu-compari
| |
| 10:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales, RANDOM]
| HDFS partitions=1831/1831 files=1831 size=150.82MB
-| runtime filters: RF007[min_max] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_catalog_page_sk, RF006[bloom] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_catalog_page_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_sold_date_sk
+| runtime filters: RF005[min_max] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_catalog_page_sk, RF006[bloom] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_sold_date_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.catalog_sales.cs_catalog_page_sk
| stored statistics:
| table: rows=4.32G size=280.96GB
| partitions: 1831/1831 rows=4.32G
@@ -1294,12 +1294,12 @@ max-parallelism=120 segment-costs=[2267, 130557, 33, 344492, 567467] cpu-compari
F02:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
Per-Instance Resources: mem-estimate=71.00MB mem-reservation=10.00MB thread-reservation=1
-max-parallelism=120 segment-costs=[16405398484, 4366] cpu-comparison-result=120 [max(120 (self) vs 22 (sum children))]
+max-parallelism=120 segment-costs=[5637342561, 4366] cpu-comparison-result=120 [max(120 (self) vs 22 (sum children))]
08:AGGREGATE [STREAMING]
| output: sum(sales_price), sum(profit), sum(return_amt), sum(net_loss)
| group by: s_store_id
| mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=6 row-size=92B cardinality=678 cost=4888175976
+| tuple-ids=6 row-size=92B cardinality=678 cost=19365585
| in pipelines: 02(GETNEXT), 03(GETNEXT)
|
07:HASH JOIN [INNER JOIN, BROADCAST]
@@ -1307,7 +1307,7 @@ max-parallelism=120 segment-costs=[16405398484, 4366] cpu-comparison-result=120
| hash predicates: store_sk = s_store_sk
| fk/pk conjuncts: assumed fk/pk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=2,4,5 row-size=64B cardinality=9.73G cost=4200341636
+| tuple-ids=2,4,5 row-size=64B cardinality=38.12M cost=16459018
| in pipelines: 02(GETNEXT), 03(GETNEXT), 05(OPEN)
|
|--F23:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
@@ -1342,26 +1342,26 @@ max-parallelism=120 segment-costs=[16405398484, 4366] cpu-comparison-result=120
| hash predicates: date_sk = d_date_sk
| fk/pk conjuncts: assumed fk/pk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=2,4 row-size=32B cardinality=9.50G cost=4159867458
+| tuple-ids=2,4 row-size=32B cardinality=37.24M cost=2444504544
| in pipelines: 02(GETNEXT), 03(GETNEXT), 04(OPEN)
|
|--F24:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
+| | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[25]
| JOIN BUILD
| | join-table-id=01 plan-id=02 cohort-id=01
| | build expressions: d_date_sk
| | runtime filters: RF002[bloom] <- d_date_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=15
| |
| 30:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=4 row-size=8B cardinality=7.30K cost=9700
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=4 row-size=8B cardinality=15 cost=10
| | in pipelines: 04(GETNEXT)
| |
| F03:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
+| max-parallelism=1 segment-costs=[12520]
| 04:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| HDFS partitions=1/1 files=1 size=2.17MB
| predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
@@ -1372,7 +1372,7 @@ max-parallelism=120 segment-costs=[16405398484, 4366] cpu-comparison-result=120
| parquet statistics predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
| parquet dictionary predicates: d_date <= DATE '2000-09-02', d_date >= DATE '2000-08-19'
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=4 row-size=8B cardinality=7.30K cost=12520
+| tuple-ids=4 row-size=8B cardinality=15 cost=12520
| in pipelines: 04(GETNEXT)
|
01:UNION
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q12.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q12.test
index 5e8366775..3673126ee 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q12.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q12.test
@@ -35,10 +35,10 @@ limit 100;
# end query 12 in stream 0 using template query12.tpl
---- PLAN
Max Per-Host Resource Reservation: Memory=76.06MB Threads=1
-Per-Host Resource Estimates: Memory=128.01GB
+Per-Host Resource Estimates: Memory=2.64GB
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=128.01GB mem-reservation=76.06MB thread-reservation=1 runtime-filters-memory=2.00MB
-| max-parallelism=1 segment-costs=[4763981234, 5850261284, 11335982837, 700]
+| Per-Instance Resources: mem-estimate=2.64GB mem-reservation=76.06MB thread-reservation=1 runtime-filters-memory=2.00MB
+| max-parallelism=1 segment-costs=[99642501, 99428779, 156178681, 700]
PLAN-ROOT SINK
| output exprs: i_item_id, i_item_desc, i_category, i_class, i_current_price, sum(ws_ext_sales_price), sum(ws_ext_sales_price) * CAST(100 AS DECIMAL(3,0)) / sum(sum(ws_ext_sales_price))
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=700
@@ -46,56 +46,35 @@ PLAN-ROOT SINK
08:TOP-N [LIMIT=100]
| order by: i_category ASC, i_class ASC, i_item_id ASC, i_item_desc ASC, sum(ws_ext_sales_price) * 100 / sum(sum(ws_ext_sales_price)) ASC
| mem-estimate=20.93KB mem-reservation=0B thread-reservation=0
-| tuple-ids=5 row-size=214B cardinality=100 cost=10688316277
+| tuple-ids=5 row-size=214B cardinality=100 cost=145171190
| in pipelines: 08(GETNEXT), 06(OPEN)
|
07:ANALYTIC
| functions: sum(sum(ws_ext_sales_price))
| partition by: i_class
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=10,9 row-size=214B cardinality=647.67M cost=647666560
+| tuple-ids=10,9 row-size=214B cardinality=11.01M cost=11007491
| in pipelines: 06(GETNEXT)
|
06:SORT
| order by: i_class ASC NULLS LAST
-| mem-estimate=119.61GB mem-reservation=12.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=10 row-size=198B cardinality=647.67M cost=5850261284
+| mem-estimate=2.03GB mem-reservation=12.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=10 row-size=198B cardinality=11.01M cost=99428779
| in pipelines: 06(GETNEXT), 05(OPEN)
|
05:AGGREGATE [FINALIZE]
| output: sum(ws_ext_sales_price)
| group by: i_item_id, i_item_desc, i_category, i_class, i_current_price
-| mem-estimate=8.40GB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=647.67M cost=3822916619
+| mem-estimate=617.97MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=11.01M cost=64972816
| in pipelines: 05(GETNEXT), 00(OPEN)
|
04:HASH JOIN [INNER JOIN]
-| hash predicates: ws_sold_date_sk = d_date_sk
-| fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| runtime filters: RF000[bloom] <- d_date_sk
-| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1,2 row-size=206B cardinality=647.67M cost=283490956
-| in pipelines: 00(GETNEXT), 02(OPEN)
-|
-|--02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
-| HDFS partitions=1/1 files=1 size=2.17MB
-| predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
-| stored statistics:
-| table: rows=73.05K size=2.17MB
-| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=73.05K
-| parquet statistics predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
-| parquet dictionary predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
-| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
-| in pipelines: 02(GETNEXT)
-|
-03:HASH JOIN [INNER JOIN]
| hash predicates: ws_item_sk = i_item_sk
| fk/pk conjuncts: ws_item_sk = i_item_sk
-| runtime filters: RF002[bloom] <- i_item_sk, RF003[min_max] <- i_item_sk
+| runtime filters: RF000[bloom] <- i_item_sk, RF001[min_max] <- i_item_sk
| mem-estimate=34.00MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,1 row-size=198B cardinality=647.67M cost=283591651
+| tuple-ids=0,2,1 row-size=206B cardinality=11.01M cost=11518667
| in pipelines: 00(GETNEXT), 01(OPEN)
|
|--01:SCAN HDFS [tpcds_partitioned_parquet_snap.item]
@@ -111,23 +90,44 @@ PLAN-ROOT SINK
| tuple-ids=1 row-size=186B cardinality=108.00K cost=727024
| in pipelines: 01(GETNEXT)
|
+03:HASH JOIN [INNER JOIN]
+| hash predicates: ws_sold_date_sk = d_date_sk
+| fk/pk conjuncts: ws_sold_date_sk = d_date_sk
+| runtime filters: RF002[bloom] <- d_date_sk
+| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,2 row-size=20B cardinality=36.71M cost=16067990
+| in pipelines: 00(GETNEXT), 02(OPEN)
+|
+|--02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
+| HDFS partitions=1/1 files=1 size=2.17MB
+| predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
+| stored statistics:
+| table: rows=73.05K size=2.17MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=73.05K
+| parquet statistics predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
+| parquet dictionary predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
+| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| tuple-ids=2 row-size=8B cardinality=31 cost=12520
+| in pipelines: 02(GETNEXT)
+|
00:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales]
HDFS partitions=1824/1824 files=1824 size=84.18MB
- runtime filters: RF003[min_max] -> ws_item_sk, RF000[bloom] -> ws_sold_date_sk, RF002[bloom] -> ws_item_sk
+ runtime filters: RF001[min_max] -> ws_item_sk, RF000[bloom] -> ws_item_sk, RF002[bloom] -> ws_sold_date_sk
stored statistics:
table: rows=2.16G size=145.75GB
partitions: 1824/1824 rows=2.16G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=3.37M
+ extrapolated-rows=disabled max-scan-range-rows=3.37M est-scan-range=32(filtered from 1824)
mem-estimate=16.00MB mem-reservation=128.00KB thread-reservation=0
- tuple-ids=0 row-size=12B cardinality=647.67M(filtered from 2.16G) cost=373242464
+ tuple-ids=0 row-size=12B cardinality=36.71M(filtered from 2.16G) cost=6343484
in pipelines: 00(GETNEXT)
---- DISTRIBUTEDPLAN
-Max Per-Host Resource Reservation: Memory=1.42GB Threads=41
-Per-Host Resource Estimates: Memory=27.92GB
+Max Per-Host Resource Reservation: Memory=249.62MB Threads=11
+Per-Host Resource Estimates: Memory=849MB
F05:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=6.50MB mem-reservation=4.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[768] cpu-comparison-result=120 [max(1 (self) vs 120 (sum children))]
+| Per-Instance Resources: mem-estimate=4.42MB mem-reservation=4.00MB thread-reservation=1
+| max-parallelism=1 segment-costs=[768] cpu-comparison-result=25 [max(1 (self) vs 25 (sum children))]
PLAN-ROOT SINK
| output exprs: i_item_id, i_item_desc, i_category, i_class, i_current_price, sum(ws_ext_sales_price), sum(ws_ext_sales_price) * CAST(100 AS DECIMAL(3,0)) / sum(sum(ws_ext_sales_price))
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=700
@@ -135,124 +135,86 @@ PLAN-ROOT SINK
14:MERGING-EXCHANGE [UNPARTITIONED]
| order by: i_category ASC, i_class ASC, i_item_id ASC, i_item_desc ASC, sum(ws_ext_sales_price) * 100 / sum(sum(ws_ext_sales_price)) ASC
| limit: 100
-| mem-estimate=2.50MB mem-reservation=0B thread-reservation=0
+| mem-estimate=428.46KB mem-reservation=0B thread-reservation=0
| tuple-ids=5 row-size=214B cardinality=100 cost=68
| in pipelines: 08(GETNEXT)
|
-F04:PLAN FRAGMENT [HASH(i_class)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=1.03GB mem-reservation=16.00MB thread-reservation=1
-max-parallelism=1140 segment-costs=[6512983524, 11335982837, 592] cpu-comparison-result=120 [max(120 (self) vs 120 (sum children))]
+F04:PLAN FRAGMENT [HASH(i_class)] hosts=10 instances=20 (adjusted from 120)
+Per-Instance Resources: mem-estimate=122.03MB mem-reservation=16.00MB thread-reservation=1
+max-parallelism=20 segment-costs=[110692150, 156178681, 592] cpu-comparison-result=25 [max(20 (self) vs 25 (sum children))]
08:TOP-N [LIMIT=100]
| order by: i_category ASC, i_class ASC, i_item_id ASC, i_item_desc ASC, sum(ws_ext_sales_price) * 100 / sum(sum(ws_ext_sales_price)) ASC
| mem-estimate=20.93KB mem-reservation=0B thread-reservation=0
-| tuple-ids=5 row-size=214B cardinality=100 cost=10688316277
+| tuple-ids=5 row-size=214B cardinality=100 cost=145171190
| in pipelines: 08(GETNEXT), 06(OPEN)
|
07:ANALYTIC
| functions: sum(sum(ws_ext_sales_price))
| partition by: i_class
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=10,9 row-size=214B cardinality=647.67M cost=647666560
+| tuple-ids=10,9 row-size=214B cardinality=11.01M cost=11007491
| in pipelines: 06(GETNEXT)
|
06:SORT
| order by: i_class ASC NULLS LAST
-| mem-estimate=1020.65MB mem-reservation=12.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=10 row-size=198B cardinality=647.67M cost=5850261284
+| mem-estimate=104.08MB mem-reservation=12.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=10 row-size=198B cardinality=11.01M cost=99428779
| in pipelines: 06(GETNEXT), 12(OPEN)
|
13:EXCHANGE [HASH(i_class)]
-| mem-estimate=33.71MB mem-reservation=0B thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=647.67M cost=662722240
+| mem-estimate=13.95MB mem-reservation=0B thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=11.01M cost=11263371
| in pipelines: 12(GETNEXT)
|
-F03:PLAN FRAGMENT [HASH(i_item_id,i_item_desc,i_category,i_class,i_current_price)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=642.48MB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=870 segment-costs=[4485638859, 8604408619] cpu-comparison-result=120 [max(120 (self) vs 120 (sum children))]
+F03:PLAN FRAGMENT [HASH(i_item_id,i_item_desc,i_category,i_class,i_current_price)] hosts=10 instances=20 (adjusted from 120)
+Per-Instance Resources: mem-estimate=126.18MB mem-reservation=34.00MB thread-reservation=1
+max-parallelism=20 segment-costs=[76236187, 146237210] cpu-comparison-result=25 [max(20 (self) vs 25 (sum children))]
12:AGGREGATE [FINALIZE]
| output: sum:merge(ws_ext_sales_price)
| group by: i_item_id, i_item_desc, i_category, i_class, i_current_price
-| mem-estimate=547.65MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=647.67M cost=3822916619
+| mem-estimate=110.38MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=11.01M cost=64972816
| in pipelines: 12(GETNEXT), 00(OPEN)
|
11:EXCHANGE [HASH(i_item_id,i_item_desc,i_category,i_class,i_current_price)]
-| mem-estimate=33.71MB mem-reservation=0B thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=647.67M cost=662722240
+| mem-estimate=13.95MB mem-reservation=0B thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=11.01M cost=11263371
| in pipelines: 00(GETNEXT)
|
-F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=20 (adjusted from 120)
Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
-Per-Instance Resources: mem-estimate=641.56MB mem-reservation=34.12MB thread-reservation=1
-max-parallelism=870 segment-costs=[4763126385, 8604408619] cpu-comparison-result=120 [max(120 (self) vs 25 (sum children))]
+Per-Instance Resources: mem-estimate=114.59MB mem-reservation=34.12MB thread-reservation=1
+max-parallelism=20 segment-costs=[98794926, 146237210] cpu-comparison-result=25 [max(20 (self) vs 25 (sum children))]
05:AGGREGATE [STREAMING]
| output: sum(ws_ext_sales_price)
| group by: i_item_id, i_item_desc, i_category, i_class, i_current_price
-| mem-estimate=530.74MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=647.67M cost=3822916619
+| mem-estimate=82.78MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=11.01M cost=64972816
| in pipelines: 00(GETNEXT)
|
04:HASH JOIN [INNER JOIN, BROADCAST]
| hash-table-id=00
-| hash predicates: ws_sold_date_sk = d_date_sk
-| fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1,2 row-size=206B cardinality=647.67M cost=283483651
-| in pipelines: 00(GETNEXT), 02(OPEN)
-|
-|--F06:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
-| JOIN BUILD
-| | join-table-id=00 plan-id=01 cohort-id=01
-| | build expressions: d_date_sk
-| | runtime filters: RF000[bloom] <- d_date_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
-| |
-| 10:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=2 row-size=8B cardinality=7.30K cost=9700
-| | in pipelines: 02(GETNEXT)
-| |
-| F02:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
-| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| HDFS partitions=1/1 files=1 size=2.17MB
-| predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
-| stored statistics:
-| table: rows=73.05K size=2.17MB
-| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=73.05K
-| parquet statistics predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
-| parquet dictionary predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
-| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
-| in pipelines: 02(GETNEXT)
-|
-03:HASH JOIN [INNER JOIN, BROADCAST]
-| hash-table-id=01
| hash predicates: ws_item_sk = i_item_sk
| fk/pk conjuncts: ws_item_sk = i_item_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,1 row-size=198B cardinality=647.67M cost=283483651
+| tuple-ids=0,2,1 row-size=206B cardinality=11.01M cost=11410667
| in pipelines: 00(GETNEXT), 01(OPEN)
|
-|--F07:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=419.74MB mem-reservation=409.00MB thread-reservation=1 runtime-filters-memory=1.00MB
+|--F06:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
+| | Per-Instance Resources: mem-estimate=79.74MB mem-reservation=69.00MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[251530]
| JOIN BUILD
-| | join-table-id=01 plan-id=02 cohort-id=01
+| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: i_item_sk
-| | runtime filters: RF002[bloom] <- i_item_sk, RF003[min_max] <- i_item_sk
-| | mem-estimate=408.00MB mem-reservation=408.00MB spill-buffer=2.00MB thread-reservation=0 cost=108000
+| | runtime filters: RF000[bloom] <- i_item_sk, RF001[min_max] <- i_item_sk
+| | mem-estimate=68.00MB mem-reservation=68.00MB spill-buffer=2.00MB thread-reservation=0 cost=108000
| |
-| 09:EXCHANGE [BROADCAST]
+| 10:EXCHANGE [BROADCAST]
| | mem-estimate=10.74MB mem-reservation=0B thread-reservation=0
| | tuple-ids=1 row-size=186B cardinality=108.00K cost=143530
| | in pipelines: 01(GETNEXT)
| |
-| F01:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
+| F02:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
| Per-Instance Resources: mem-estimate=16.74MB mem-reservation=512.00KB thread-reservation=1
| max-parallelism=4 segment-costs=[783679]
| 01:SCAN HDFS [tpcds_partitioned_parquet_snap.item, RANDOM]
@@ -268,23 +230,61 @@ max-parallelism=870 segment-costs=[4763126385, 8604408619] cpu-comparison-result
| tuple-ids=1 row-size=186B cardinality=108.00K cost=727024
| in pipelines: 01(GETNEXT)
|
+03:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=01
+| hash predicates: ws_sold_date_sk = d_date_sk
+| fk/pk conjuncts: ws_sold_date_sk = d_date_sk
+| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,2 row-size=20B cardinality=36.71M cost=16067959
+| in pipelines: 00(GETNEXT), 02(OPEN)
+|
+|--F07:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
+| | Per-Instance Resources: mem-estimate=4.89MB mem-reservation=4.88MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[71]
+| JOIN BUILD
+| | join-table-id=01 plan-id=02 cohort-id=01
+| | build expressions: d_date_sk
+| | runtime filters: RF002[bloom] <- d_date_sk
+| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0 cost=31
+| |
+| 09:EXCHANGE [BROADCAST]
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=2 row-size=8B cardinality=31 cost=40
+| | in pipelines: 02(GETNEXT)
+| |
+| F01:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| max-parallelism=1 segment-costs=[12521]
+| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| HDFS partitions=1/1 files=1 size=2.17MB
+| predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
+| stored statistics:
+| table: rows=73.05K size=2.17MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=73.05K
+| parquet statistics predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
+| parquet dictionary predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
+| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| tuple-ids=2 row-size=8B cardinality=31 cost=12520
+| in pipelines: 02(GETNEXT)
+|
00:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales, RANDOM]
HDFS partitions=1824/1824 files=1824 size=84.18MB
- runtime filters: RF003[min_max] -> ws_item_sk, RF000[bloom] -> ws_sold_date_sk, RF002[bloom] -> ws_item_sk
+ runtime filters: RF001[min_max] -> ws_item_sk, RF000[bloom] -> ws_item_sk, RF002[bloom] -> ws_sold_date_sk
stored statistics:
table: rows=2.16G size=145.75GB
partitions: 1824/1824 rows=2.16G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=3.37M
+ extrapolated-rows=disabled max-scan-range-rows=3.37M est-scan-range=32(filtered from 1824)
mem-estimate=16.00MB mem-reservation=128.00KB thread-reservation=0
- tuple-ids=0 row-size=12B cardinality=647.67M(filtered from 2.16G) cost=373242464
+ tuple-ids=0 row-size=12B cardinality=36.71M(filtered from 2.16G) cost=6343484
in pipelines: 00(GETNEXT)
---- PARALLELPLANS
-Max Per-Host Resource Reservation: Memory=1.42GB Threads=41
-Per-Host Resource Estimates: Memory=27.92GB
+Max Per-Host Resource Reservation: Memory=249.62MB Threads=11
+Per-Host Resource Estimates: Memory=849MB
F05:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=6.50MB mem-reservation=4.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[768] cpu-comparison-result=120 [max(1 (self) vs 120 (sum children))]
+| Per-Instance Resources: mem-estimate=4.42MB mem-reservation=4.00MB thread-reservation=1
+| max-parallelism=1 segment-costs=[768] cpu-comparison-result=25 [max(1 (self) vs 25 (sum children))]
PLAN-ROOT SINK
| output exprs: i_item_id, i_item_desc, i_category, i_class, i_current_price, sum(ws_ext_sales_price), sum(ws_ext_sales_price) * CAST(100 AS DECIMAL(3,0)) / sum(sum(ws_ext_sales_price))
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=700
@@ -292,124 +292,86 @@ PLAN-ROOT SINK
14:MERGING-EXCHANGE [UNPARTITIONED]
| order by: i_category ASC, i_class ASC, i_item_id ASC, i_item_desc ASC, sum(ws_ext_sales_price) * 100 / sum(sum(ws_ext_sales_price)) ASC
| limit: 100
-| mem-estimate=2.50MB mem-reservation=0B thread-reservation=0
+| mem-estimate=428.46KB mem-reservation=0B thread-reservation=0
| tuple-ids=5 row-size=214B cardinality=100 cost=68
| in pipelines: 08(GETNEXT)
|
-F04:PLAN FRAGMENT [HASH(i_class)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=1.03GB mem-reservation=16.00MB thread-reservation=1
-max-parallelism=1140 segment-costs=[6512983524, 11335982837, 592] cpu-comparison-result=120 [max(120 (self) vs 120 (sum children))]
+F04:PLAN FRAGMENT [HASH(i_class)] hosts=10 instances=20 (adjusted from 120)
+Per-Instance Resources: mem-estimate=122.03MB mem-reservation=16.00MB thread-reservation=1
+max-parallelism=20 segment-costs=[110692150, 156178681, 592] cpu-comparison-result=25 [max(20 (self) vs 25 (sum children))]
08:TOP-N [LIMIT=100]
| order by: i_category ASC, i_class ASC, i_item_id ASC, i_item_desc ASC, sum(ws_ext_sales_price) * 100 / sum(sum(ws_ext_sales_price)) ASC
| mem-estimate=20.93KB mem-reservation=0B thread-reservation=0
-| tuple-ids=5 row-size=214B cardinality=100 cost=10688316277
+| tuple-ids=5 row-size=214B cardinality=100 cost=145171190
| in pipelines: 08(GETNEXT), 06(OPEN)
|
07:ANALYTIC
| functions: sum(sum(ws_ext_sales_price))
| partition by: i_class
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=10,9 row-size=214B cardinality=647.67M cost=647666560
+| tuple-ids=10,9 row-size=214B cardinality=11.01M cost=11007491
| in pipelines: 06(GETNEXT)
|
06:SORT
| order by: i_class ASC NULLS LAST
-| mem-estimate=1020.65MB mem-reservation=12.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=10 row-size=198B cardinality=647.67M cost=5850261284
+| mem-estimate=104.08MB mem-reservation=12.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=10 row-size=198B cardinality=11.01M cost=99428779
| in pipelines: 06(GETNEXT), 12(OPEN)
|
13:EXCHANGE [HASH(i_class)]
-| mem-estimate=33.71MB mem-reservation=0B thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=647.67M cost=662722240
+| mem-estimate=13.95MB mem-reservation=0B thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=11.01M cost=11263371
| in pipelines: 12(GETNEXT)
|
-F03:PLAN FRAGMENT [HASH(i_item_id,i_item_desc,i_category,i_class,i_current_price)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=642.48MB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=870 segment-costs=[4485638859, 8604408619] cpu-comparison-result=120 [max(120 (self) vs 120 (sum children))]
+F03:PLAN FRAGMENT [HASH(i_item_id,i_item_desc,i_category,i_class,i_current_price)] hosts=10 instances=20 (adjusted from 120)
+Per-Instance Resources: mem-estimate=126.18MB mem-reservation=34.00MB thread-reservation=1
+max-parallelism=20 segment-costs=[76236187, 146237210] cpu-comparison-result=25 [max(20 (self) vs 25 (sum children))]
12:AGGREGATE [FINALIZE]
| output: sum:merge(ws_ext_sales_price)
| group by: i_item_id, i_item_desc, i_category, i_class, i_current_price
-| mem-estimate=547.65MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=647.67M cost=3822916619
+| mem-estimate=110.38MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=11.01M cost=64972816
| in pipelines: 12(GETNEXT), 00(OPEN)
|
11:EXCHANGE [HASH(i_item_id,i_item_desc,i_category,i_class,i_current_price)]
-| mem-estimate=33.71MB mem-reservation=0B thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=647.67M cost=662722240
+| mem-estimate=13.95MB mem-reservation=0B thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=11.01M cost=11263371
| in pipelines: 00(GETNEXT)
|
-F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=20 (adjusted from 120)
Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
-Per-Instance Resources: mem-estimate=641.56MB mem-reservation=34.12MB thread-reservation=1
-max-parallelism=870 segment-costs=[4763126385, 8604408619] cpu-comparison-result=120 [max(120 (self) vs 25 (sum children))]
+Per-Instance Resources: mem-estimate=114.59MB mem-reservation=34.12MB thread-reservation=1
+max-parallelism=20 segment-costs=[98794926, 146237210] cpu-comparison-result=25 [max(20 (self) vs 25 (sum children))]
05:AGGREGATE [STREAMING]
| output: sum(ws_ext_sales_price)
| group by: i_item_id, i_item_desc, i_category, i_class, i_current_price
-| mem-estimate=530.74MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=647.67M cost=3822916619
+| mem-estimate=82.78MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=11.01M cost=64972816
| in pipelines: 00(GETNEXT)
|
04:HASH JOIN [INNER JOIN, BROADCAST]
| hash-table-id=00
-| hash predicates: ws_sold_date_sk = d_date_sk
-| fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1,2 row-size=206B cardinality=647.67M cost=283483651
-| in pipelines: 00(GETNEXT), 02(OPEN)
-|
-|--F06:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
-| JOIN BUILD
-| | join-table-id=00 plan-id=01 cohort-id=01
-| | build expressions: d_date_sk
-| | runtime filters: RF000[bloom] <- d_date_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
-| |
-| 10:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=2 row-size=8B cardinality=7.30K cost=9700
-| | in pipelines: 02(GETNEXT)
-| |
-| F02:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
-| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| HDFS partitions=1/1 files=1 size=2.17MB
-| predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
-| stored statistics:
-| table: rows=73.05K size=2.17MB
-| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=73.05K
-| parquet statistics predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
-| parquet dictionary predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
-| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
-| in pipelines: 02(GETNEXT)
-|
-03:HASH JOIN [INNER JOIN, BROADCAST]
-| hash-table-id=01
| hash predicates: ws_item_sk = i_item_sk
| fk/pk conjuncts: ws_item_sk = i_item_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,1 row-size=198B cardinality=647.67M cost=283483651
+| tuple-ids=0,2,1 row-size=206B cardinality=11.01M cost=11410667
| in pipelines: 00(GETNEXT), 01(OPEN)
|
-|--F07:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=419.74MB mem-reservation=409.00MB thread-reservation=1 runtime-filters-memory=1.00MB
+|--F06:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
+| | Per-Instance Resources: mem-estimate=79.74MB mem-reservation=69.00MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[251530]
| JOIN BUILD
-| | join-table-id=01 plan-id=02 cohort-id=01
+| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: i_item_sk
-| | runtime filters: RF002[bloom] <- i_item_sk, RF003[min_max] <- i_item_sk
-| | mem-estimate=408.00MB mem-reservation=408.00MB spill-buffer=2.00MB thread-reservation=0 cost=108000
+| | runtime filters: RF000[bloom] <- i_item_sk, RF001[min_max] <- i_item_sk
+| | mem-estimate=68.00MB mem-reservation=68.00MB spill-buffer=2.00MB thread-reservation=0 cost=108000
| |
-| 09:EXCHANGE [BROADCAST]
+| 10:EXCHANGE [BROADCAST]
| | mem-estimate=10.74MB mem-reservation=0B thread-reservation=0
| | tuple-ids=1 row-size=186B cardinality=108.00K cost=143530
| | in pipelines: 01(GETNEXT)
| |
-| F01:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
+| F02:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
| Per-Instance Resources: mem-estimate=16.74MB mem-reservation=512.00KB thread-reservation=1
| max-parallelism=4 segment-costs=[783679]
| 01:SCAN HDFS [tpcds_partitioned_parquet_snap.item, RANDOM]
@@ -425,15 +387,53 @@ max-parallelism=870 segment-costs=[4763126385, 8604408619] cpu-comparison-result
| tuple-ids=1 row-size=186B cardinality=108.00K cost=727024
| in pipelines: 01(GETNEXT)
|
+03:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=01
+| hash predicates: ws_sold_date_sk = d_date_sk
+| fk/pk conjuncts: ws_sold_date_sk = d_date_sk
+| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,2 row-size=20B cardinality=36.71M cost=16067959
+| in pipelines: 00(GETNEXT), 02(OPEN)
+|
+|--F07:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
+| | Per-Instance Resources: mem-estimate=4.89MB mem-reservation=4.88MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[71]
+| JOIN BUILD
+| | join-table-id=01 plan-id=02 cohort-id=01
+| | build expressions: d_date_sk
+| | runtime filters: RF002[bloom] <- d_date_sk
+| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0 cost=31
+| |
+| 09:EXCHANGE [BROADCAST]
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=2 row-size=8B cardinality=31 cost=40
+| | in pipelines: 02(GETNEXT)
+| |
+| F01:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| max-parallelism=1 segment-costs=[12521]
+| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| HDFS partitions=1/1 files=1 size=2.17MB
+| predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
+| stored statistics:
+| table: rows=73.05K size=2.17MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=73.05K
+| parquet statistics predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
+| parquet dictionary predicates: d_date <= DATE '1998-04-27', d_date >= DATE '1998-03-28'
+| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| tuple-ids=2 row-size=8B cardinality=31 cost=12520
+| in pipelines: 02(GETNEXT)
+|
00:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales, RANDOM]
HDFS partitions=1824/1824 files=1824 size=84.18MB
- runtime filters: RF003[min_max] -> ws_item_sk, RF000[bloom] -> ws_sold_date_sk, RF002[bloom] -> ws_item_sk
+ runtime filters: RF001[min_max] -> ws_item_sk, RF000[bloom] -> ws_item_sk, RF002[bloom] -> ws_sold_date_sk
stored statistics:
table: rows=2.16G size=145.75GB
partitions: 1824/1824 rows=2.16G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=3.37M
+ extrapolated-rows=disabled max-scan-range-rows=3.37M est-scan-range=32(filtered from 1824)
mem-estimate=16.00MB mem-reservation=128.00KB thread-reservation=0
- tuple-ids=0 row-size=12B cardinality=647.67M(filtered from 2.16G) cost=373242464
+ tuple-ids=0 row-size=12B cardinality=36.71M(filtered from 2.16G) cost=6343484
in pipelines: 00(GETNEXT)
====
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q16.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q16.test
index 7bc1d6772..0bcb1fe84 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q16.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q16.test
@@ -31,11 +31,11 @@ limit 100;
# end query 16 in stream 0 using template query16.tpl
---- PLAN
-Max Per-Host Resource Reservation: Memory=80.03MB Threads=1
-Per-Host Resource Estimates: Memory=5.32GB
+Max Per-Host Resource Reservation: Memory=73.03MB Threads=1
+Per-Host Resource Estimates: Memory=206MB
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=5.32GB mem-reservation=80.03MB thread-reservation=1 runtime-filters-memory=11.00MB
-| max-parallelism=1 segment-costs=[4199617701, 4869719, 0, 0]
+| Per-Instance Resources: mem-estimate=206.13MB mem-reservation=73.03MB thread-reservation=1 runtime-filters-memory=4.00MB
+| max-parallelism=1 segment-costs=[4068221571, 153674, 0, 0]
PLAN-ROOT SINK
| output exprs: count(cs_order_number), sum(cs_ext_ship_cost), sum(cs_net_profit)
| mem-estimate=0B mem-reservation=0B thread-reservation=0 cost=0
@@ -49,57 +49,36 @@ PLAN-ROOT SINK
12:AGGREGATE [FINALIZE]
| output: count(cs_order_number), sum:merge(cs_ext_ship_cost), sum:merge(cs_net_profit)
| mem-estimate=16.00KB mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=9 row-size=40B cardinality=1 cost=4869719
+| tuple-ids=9 row-size=40B cardinality=1 cost=153674
| in pipelines: 12(GETNEXT), 11(OPEN)
|
11:AGGREGATE
| output: sum(cs_ext_ship_cost), sum(cs_net_profit)
| group by: cs_order_number
-| mem-estimate=1.12GB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=8 row-size=40B cardinality=23.19M cost=67386322
+| mem-estimate=36.29MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=8 row-size=40B cardinality=731.78K cost=2171682
| in pipelines: 11(GETNEXT), 05(OPEN)
|
10:HASH JOIN [RIGHT ANTI JOIN]
| hash predicates: cr1.cr_order_number = cs1.cs_order_number
-| mem-estimate=2.64GB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,2,3,1 row-size=88B cardinality=23.19M cost=138203372
+| mem-estimate=85.06MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=0,2,1,3 row-size=88B cardinality=731.78K cost=111676741
| in pipelines: 05(GETNEXT), 04(OPEN)
|
|--09:HASH JOIN [RIGHT SEMI JOIN]
| | hash predicates: cs2.cs_order_number = cs1.cs_order_number
| | other join predicates: cs1.cs_warehouse_sk != cs2.cs_warehouse_sk
| | runtime filters: RF000[bloom] <- cs1.cs_order_number, RF001[min_max] <- cs1.cs_order_number
-| | mem-estimate=2.64GB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| | tuple-ids=0,2,3,1 row-size=88B cardinality=23.19M cost=1135491016
+| | mem-estimate=85.06MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| | tuple-ids=0,2,1,3 row-size=88B cardinality=731.78K cost=1108964384
| | in pipelines: 04(GETNEXT), 00(OPEN)
| |
| |--08:HASH JOIN [INNER JOIN]
-| | | hash predicates: cs1.cs_ship_date_sk = d_date_sk
-| | | fk/pk conjuncts: cs1.cs_ship_date_sk = d_date_sk
-| | | runtime filters: RF002[bloom] <- d_date_sk, RF003[min_max] <- d_date_sk
-| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=0,2,3,1 row-size=88B cardinality=23.19M cost=10157192
-| | | in pipelines: 00(GETNEXT), 01(OPEN)
-| | |
-| | |--01:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
-| | | HDFS partitions=1/1 files=1 size=2.17MB
-| | | predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
-| | | stored statistics:
-| | | table: rows=73.05K size=2.17MB
-| | | columns: all
-| | | extrapolated-rows=disabled max-scan-range-rows=73.05K
-| | | parquet statistics predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
-| | | parquet dictionary predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
-| | | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | | tuple-ids=1 row-size=8B cardinality=7.30K cost=12520
-| | | in pipelines: 01(GETNEXT)
-| | |
-| | 07:HASH JOIN [INNER JOIN]
| | | hash predicates: cs1.cs_call_center_sk = cc_call_center_sk
| | | fk/pk conjuncts: cs1.cs_call_center_sk = cc_call_center_sk
-| | | runtime filters: RF004[bloom] <- cc_call_center_sk, RF005[min_max] <- cc_call_center_sk
+| | | runtime filters: RF002[bloom] <- cc_call_center_sk, RF003[min_max] <- cc_call_center_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=0,2,3 row-size=80B cardinality=23.19M cost=24595079
+| | | tuple-ids=0,2,1,3 row-size=88B cardinality=731.78K cost=776164
| | | in pipelines: 00(GETNEXT), 03(OPEN)
| | |
| | |--03:SCAN HDFS [tpcds_partitioned_parquet_snap.call_center]
@@ -115,6 +94,27 @@ PLAN-ROOT SINK
| | | tuple-ids=3 row-size=30B cardinality=14 cost=13
| | | in pipelines: 03(GETNEXT)
| | |
+| | 07:HASH JOIN [INNER JOIN]
+| | | hash predicates: cs1.cs_ship_date_sk = d_date_sk
+| | | fk/pk conjuncts: cs1.cs_ship_date_sk = d_date_sk
+| | | runtime filters: RF004[bloom] <- d_date_sk, RF005[min_max] <- d_date_sk
+| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| | | tuple-ids=0,2,1 row-size=58B cardinality=2.51M cost=20847880
+| | | in pipelines: 00(GETNEXT), 01(OPEN)
+| | |
+| | |--01:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
+| | | HDFS partitions=1/1 files=1 size=2.17MB
+| | | predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
+| | | stored statistics:
+| | | table: rows=73.05K size=2.17MB
+| | | columns: all
+| | | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | | parquet statistics predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
+| | | parquet dictionary predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
+| | | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | | tuple-ids=1 row-size=8B cardinality=61 cost=12520
+| | | in pipelines: 01(GETNEXT)
+| | |
| | 06:HASH JOIN [INNER JOIN]
| | | hash predicates: cs1.cs_ship_addr_sk = ca_address_sk
| | | fk/pk conjuncts: cs1.cs_ship_addr_sk = ca_address_sk
@@ -138,7 +138,7 @@ PLAN-ROOT SINK
| | |
| | 00:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales cs1]
| | HDFS partitions=1831/1831 files=1831 size=150.82MB
-| | runtime filters: RF003[min_max] -> cs1.cs_ship_date_sk, RF005[min_max] -> cs1.cs_call_center_sk, RF007[min_max] -> cs1.cs_ship_addr_sk, RF002[bloom] -> cs1.cs_ship_date_sk, RF004[bloom] -> cs1.cs_call_center_sk, RF006[bloom] -> cs1.cs_ship_addr_sk
+| | runtime filters: RF003[min_max] -> cs1.cs_call_center_sk, RF005[min_max] -> cs1.cs_ship_date_sk, RF007[min_max] -> cs1.cs_ship_addr_sk, RF002[bloom] -> cs1.cs_call_center_sk, RF004[bloom] -> cs1.cs_ship_date_sk, RF006[bloom] -> cs1.cs_ship_addr_sk
| | stored statistics:
| | table: rows=4.32G size=280.96GB
| | partitions: 1831/1831 rows=4.32G
@@ -171,11 +171,11 @@ PLAN-ROOT SINK
tuple-ids=6 row-size=8B cardinality=432.02M cost=49768357
in pipelines: 05(GETNEXT)
---- DISTRIBUTEDPLAN
-Max Per-Host Resource Reservation: Memory=619.80MB Threads=45
-Per-Host Resource Estimates: Memory=1.73GB
+Max Per-Host Resource Reservation: Memory=324.55MB Threads=42
+Per-Host Resource Estimates: Memory=1.00GB
F07:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=32.00KB mem-reservation=0B thread-reservation=1
-| max-parallelism=1 segment-costs=[0, 0, 0] cpu-comparison-result=150 [max(1 (self) vs 150 (sum children))]
+| max-parallelism=1 segment-costs=[0, 0, 0] cpu-comparison-result=140 [max(1 (self) vs 140 (sum children))]
PLAN-ROOT SINK
| output exprs: count(cs_order_number), sum(cs_ext_ship_cost), sum(cs_net_profit)
| mem-estimate=0B mem-reservation=0B thread-reservation=0 cost=0
@@ -197,124 +197,86 @@ PLAN-ROOT SINK
| tuple-ids=9 row-size=40B cardinality=1 cost=0
| in pipelines: 12(GETNEXT)
|
-F06:PLAN FRAGMENT [HASH(cs1.cs_order_number)] hosts=10 instances=30 (adjusted from 120)
-Per-Instance Resources: mem-estimate=48.92MB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=30 segment-costs=[247761556, 4869719, 3] cpu-comparison-result=150 [max(80 (self) vs 150 (sum children))]
+F06:PLAN FRAGMENT [HASH(cs1.cs_order_number)] hosts=10 instances=20 (adjusted from 120)
+Per-Instance Resources: mem-estimate=20.59MB mem-reservation=2.88MB thread-reservation=1
+max-parallelism=20 segment-costs=[177013385, 153674, 3] cpu-comparison-result=140 [max(70 (self) vs 140 (sum children))]
12:AGGREGATE
| output: count(cs_order_number), sum:merge(cs_ext_ship_cost), sum:merge(cs_net_profit)
| mem-estimate=16.00KB mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=9 row-size=40B cardinality=1 cost=4869719
+| tuple-ids=9 row-size=40B cardinality=1 cost=153674
| in pipelines: 12(GETNEXT), 11(OPEN)
|
11:AGGREGATE
| output: sum(cs_ext_ship_cost), sum(cs_net_profit)
| group by: cs_order_number
-| mem-estimate=38.33MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=8 row-size=40B cardinality=23.19M cost=68852014
+| mem-estimate=10.00MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
+| tuple-ids=8 row-size=40B cardinality=731.78K cost=2173116
| in pipelines: 11(GETNEXT), 05(OPEN)
|
10:HASH JOIN [RIGHT ANTI JOIN, PARTITIONED]
| hash-table-id=00
| hash predicates: cr1.cr_order_number = cs1.cs_order_number
-| mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,2,3,1 row-size=88B cardinality=23.19M cost=115014230
+| mem-estimate=0B mem-reservation=0B spill-buffer=512.00KB thread-reservation=0
+| tuple-ids=0,2,1,3 row-size=88B cardinality=731.78K cost=110944957
| in pipelines: 05(GETNEXT), 04(OPEN)
|
-|--F08:PLAN FRAGMENT [HASH(cs1.cs_order_number)] hosts=10 instances=30 (adjusted from 120)
-| | Per-Instance Resources: mem-estimate=101.98MB mem-reservation=34.00MB thread-reservation=1
-| | max-parallelism=30 segment-costs=[1853919983] cpu-comparison-result=150 [max(150 (self) vs 150 (sum children))]
+|--F08:PLAN FRAGMENT [HASH(cs1.cs_order_number)] hosts=10 instances=20 (adjusted from 120)
+| | Per-Instance Resources: mem-estimate=20.38MB mem-reservation=8.50MB thread-reservation=1
+| | max-parallelism=20 segment-costs=[1827393351] cpu-comparison-result=140 [max(140 (self) vs 140 (sum children))]
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: cs1.cs_order_number
-| | mem-estimate=90.10MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 cost=23189142
+| | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0 cost=731784
| |
| 09:HASH JOIN [RIGHT SEMI JOIN, PARTITIONED]
| | hash-table-id=01
| | hash predicates: cs2.cs_order_number = cs1.cs_order_number
| | other join predicates: cs1.cs_warehouse_sk != cs2.cs_warehouse_sk
-| | mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| | tuple-ids=0,2,3,1 row-size=88B cardinality=23.19M cost=1112301874
+| | mem-estimate=0B mem-reservation=0B spill-buffer=512.00KB thread-reservation=0
+| | tuple-ids=0,2,1,3 row-size=88B cardinality=731.78K cost=1108232600
| | in pipelines: 04(GETNEXT), 00(OPEN)
| |
-| |--F09:PLAN FRAGMENT [HASH(cs1.cs_order_number)] hosts=10 instances=30 (adjusted from 120)
-| | | Per-Instance Resources: mem-estimate=120.23MB mem-reservation=42.00MB thread-reservation=1 runtime-filters-memory=8.00MB
-| | | max-parallelism=30 segment-costs=[37659166] cpu-comparison-result=150 [max(150 (self) vs 42 (sum children))]
+| |--F09:PLAN FRAGMENT [HASH(cs1.cs_order_number)] hosts=10 instances=20 (adjusted from 120)
+| | | Per-Instance Resources: mem-estimate=27.74MB mem-reservation=9.50MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=20 segment-costs=[1188417] cpu-comparison-result=140 [max(140 (self) vs 42 (sum children))]
| | JOIN BUILD
| | | join-table-id=01 plan-id=02 cohort-id=02
| | | build expressions: cs1.cs_order_number
| | | runtime filters: RF000[bloom] <- cs1.cs_order_number, RF001[min_max] <- cs1.cs_order_number
-| | | mem-estimate=90.10MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 cost=23189142
+| | | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0 cost=731784
| | |
| | 18:EXCHANGE [HASH(cs1.cs_order_number)]
-| | | mem-estimate=22.13MB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=0,2,3,1 row-size=88B cardinality=23.19M cost=14470024
+| | | mem-estimate=18.24MB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=0,2,1,3 row-size=88B cardinality=731.78K cost=456633
| | | in pipelines: 00(GETNEXT)
| | |
| | F02:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
| | Per-Host Shared Resources: mem-estimate=3.00MB mem-reservation=3.00MB thread-reservation=0 runtime-filters-memory=3.00MB
-| | Per-Instance Resources: mem-estimate=28.13MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=230 segment-costs=[2238695495]
+| | Per-Instance Resources: mem-estimate=24.09MB mem-reservation=1.00MB thread-reservation=1
+| | max-parallelism=210 segment-costs=[2052747176]
| | 08:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash-table-id=02
-| | | hash predicates: cs1.cs_ship_date_sk = d_date_sk
-| | | fk/pk conjuncts: cs1.cs_ship_date_sk = d_date_sk
-| | | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=0,2,3,1 row-size=88B cardinality=23.19M cost=10149887
-| | | in pipelines: 00(GETNEXT), 01(OPEN)
-| | |
-| | |--F10:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | | max-parallelism=10 segment-costs=[17005]
-| | | JOIN BUILD
-| | | | join-table-id=02 plan-id=03 cohort-id=03
-| | | | build expressions: d_date_sk
-| | | | runtime filters: RF002[bloom] <- d_date_sk, RF003[min_max] <- d_date_sk
-| | | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
-| | | |
-| | | 16:EXCHANGE [BROADCAST]
-| | | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | | tuple-ids=1 row-size=8B cardinality=7.30K cost=9700
-| | | | in pipelines: 01(GETNEXT)
-| | | |
-| | | F05:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | | max-parallelism=1 segment-costs=[12835]
-| | | 01:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| | | HDFS partitions=1/1 files=1 size=2.17MB
-| | | predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
-| | | stored statistics:
-| | | table: rows=73.05K size=2.17MB
-| | | columns: all
-| | | extrapolated-rows=disabled max-scan-range-rows=73.05K
-| | | parquet statistics predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
-| | | parquet dictionary predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
-| | | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | | tuple-ids=1 row-size=8B cardinality=7.30K cost=12520
-| | | in pipelines: 01(GETNEXT)
-| | |
-| | 07:HASH JOIN [INNER JOIN, BROADCAST]
-| | | hash-table-id=03
| | | hash predicates: cs1.cs_call_center_sk = cc_call_center_sk
| | | fk/pk conjuncts: cs1.cs_call_center_sk = cc_call_center_sk
| | | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=0,2,3 row-size=80B cardinality=23.19M cost=24595065
+| | | tuple-ids=0,2,1,3 row-size=88B cardinality=731.78K cost=776150
| | | in pipelines: 00(GETNEXT), 03(OPEN)
| | |
-| | |--F11:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
+| | |--F10:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
| | | | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
| | | | max-parallelism=10 segment-costs=[24]
| | | JOIN BUILD
-| | | | join-table-id=03 plan-id=04 cohort-id=03
+| | | | join-table-id=02 plan-id=03 cohort-id=03
| | | | build expressions: cc_call_center_sk
-| | | | runtime filters: RF004[bloom] <- cc_call_center_sk, RF005[min_max] <- cc_call_center_sk
+| | | | runtime filters: RF002[bloom] <- cc_call_center_sk, RF003[min_max] <- cc_call_center_sk
| | | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=14
| | | |
-| | | 15:EXCHANGE [BROADCAST]
+| | | 16:EXCHANGE [BROADCAST]
| | | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
| | | | tuple-ids=3 row-size=30B cardinality=14 cost=10
| | | | in pipelines: 03(GETNEXT)
| | | |
-| | | F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | | F05:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| | | Per-Instance Resources: mem-estimate=16.13MB mem-reservation=16.00KB thread-reservation=1
| | | max-parallelism=1 segment-costs=[14]
| | | 03:SCAN HDFS [tpcds_partitioned_parquet_snap.call_center, RANDOM]
@@ -330,6 +292,44 @@ max-parallelism=30 segment-costs=[247761556, 4869719, 3] cpu-comparison-result=1
| | | tuple-ids=3 row-size=30B cardinality=14 cost=13
| | | in pipelines: 03(GETNEXT)
| | |
+| | 07:HASH JOIN [INNER JOIN, BROADCAST]
+| | | hash-table-id=03
+| | | hash predicates: cs1.cs_ship_date_sk = d_date_sk
+| | | fk/pk conjuncts: cs1.cs_ship_date_sk = d_date_sk
+| | | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| | | tuple-ids=0,2,1 row-size=58B cardinality=2.51M cost=20847819
+| | | in pipelines: 00(GETNEXT), 01(OPEN)
+| | |
+| | |--F11:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
+| | | | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | | max-parallelism=10 segment-costs=[141]
+| | | JOIN BUILD
+| | | | join-table-id=03 plan-id=04 cohort-id=03
+| | | | build expressions: d_date_sk
+| | | | runtime filters: RF004[bloom] <- d_date_sk, RF005[min_max] <- d_date_sk
+| | | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=61
+| | | |
+| | | 15:EXCHANGE [BROADCAST]
+| | | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | | tuple-ids=1 row-size=8B cardinality=61 cost=80
+| | | | in pipelines: 01(GETNEXT)
+| | | |
+| | | F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| | | max-parallelism=1 segment-costs=[12522]
+| | | 01:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| | | HDFS partitions=1/1 files=1 size=2.17MB
+| | | predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
+| | | stored statistics:
+| | | table: rows=73.05K size=2.17MB
+| | | columns: all
+| | | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | | parquet statistics predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
+| | | parquet dictionary predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
+| | | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | | tuple-ids=1 row-size=8B cardinality=61 cost=12520
+| | | in pipelines: 01(GETNEXT)
+| | |
| | 06:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash-table-id=04
| | | hash predicates: cs1.cs_ship_addr_sk = ca_address_sk
@@ -370,7 +370,7 @@ max-parallelism=30 segment-costs=[247761556, 4869719, 3] cpu-comparison-result=1
| | |
| | 00:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales cs1, RANDOM]
| | HDFS partitions=1831/1831 files=1831 size=150.82MB
-| | runtime filters: RF003[min_max] -> cs1.cs_ship_date_sk, RF005[min_max] -> cs1.cs_call_center_sk, RF007[min_max] -> cs1.cs_ship_addr_sk, RF002[bloom] -> cs1.cs_ship_date_sk, RF004[bloom] -> cs1.cs_call_center_sk, RF006[bloom] -> cs1.cs_ship_addr_sk
+| | runtime filters: RF003[min_max] -> cs1.cs_call_center_sk, RF005[min_max] -> cs1.cs_ship_date_sk, RF007[min_max] -> cs1.cs_ship_addr_sk, RF002[bloom] -> cs1.cs_call_center_sk, RF004[bloom] -> cs1.cs_ship_date_sk, RF006[bloom] -> cs1.cs_ship_addr_sk
| | stored statistics:
| | table: rows=4.32G size=280.96GB
| | partitions: 1831/1831 rows=4.32G
@@ -386,8 +386,8 @@ max-parallelism=30 segment-costs=[247761556, 4869719, 3] cpu-comparison-result=1
| | in pipelines: 04(GETNEXT)
| |
| F01:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
-| Per-Host Shared Resources: mem-estimate=8.00MB mem-reservation=8.00MB thread-reservation=0 runtime-filters-memory=8.00MB
-| Per-Instance Resources: mem-estimate=17.88MB mem-reservation=1.00MB thread-reservation=1
+| Per-Host Shared Resources: mem-estimate=1.00MB mem-reservation=1.00MB thread-reservation=0 runtime-filters-memory=1.00MB
+| Per-Instance Resources: mem-estimate=17.25MB mem-reservation=1.00MB thread-reservation=1
| max-parallelism=640 segment-costs=[6310769910]
| 04:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales cs2, RANDOM]
| HDFS partitions=1831/1831 files=1831 size=150.82MB
@@ -407,7 +407,7 @@ max-parallelism=30 segment-costs=[247761556, 4869719, 3] cpu-comparison-result=1
| in pipelines: 05(GETNEXT)
|
F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=50 (adjusted from 120)
-Per-Instance Resources: mem-estimate=17.41MB mem-reservation=32.00KB thread-reservation=1
+Per-Instance Resources: mem-estimate=16.94MB mem-reservation=32.00KB thread-reservation=1
max-parallelism=50 segment-costs=[494918664]
05:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_returns cr1, RANDOM]
HDFS partitions=2060/2060 files=2060 size=30.47MB
@@ -420,11 +420,11 @@ max-parallelism=50 segment-costs=[494918664]
tuple-ids=6 row-size=8B cardinality=432.02M cost=49768357
in pipelines: 05(GETNEXT)
---- PARALLELPLANS
-Max Per-Host Resource Reservation: Memory=619.80MB Threads=45
-Per-Host Resource Estimates: Memory=1.73GB
+Max Per-Host Resource Reservation: Memory=324.55MB Threads=42
+Per-Host Resource Estimates: Memory=1.00GB
F07:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=32.00KB mem-reservation=0B thread-reservation=1
-| max-parallelism=1 segment-costs=[0, 0, 0] cpu-comparison-result=150 [max(1 (self) vs 150 (sum children))]
+| max-parallelism=1 segment-costs=[0, 0, 0] cpu-comparison-result=140 [max(1 (self) vs 140 (sum children))]
PLAN-ROOT SINK
| output exprs: count(cs_order_number), sum(cs_ext_ship_cost), sum(cs_net_profit)
| mem-estimate=0B mem-reservation=0B thread-reservation=0 cost=0
@@ -446,124 +446,86 @@ PLAN-ROOT SINK
| tuple-ids=9 row-size=40B cardinality=1 cost=0
| in pipelines: 12(GETNEXT)
|
-F06:PLAN FRAGMENT [HASH(cs1.cs_order_number)] hosts=10 instances=30 (adjusted from 120)
-Per-Instance Resources: mem-estimate=48.92MB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=30 segment-costs=[247761556, 4869719, 3] cpu-comparison-result=150 [max(80 (self) vs 150 (sum children))]
+F06:PLAN FRAGMENT [HASH(cs1.cs_order_number)] hosts=10 instances=20 (adjusted from 120)
+Per-Instance Resources: mem-estimate=20.59MB mem-reservation=2.88MB thread-reservation=1
+max-parallelism=20 segment-costs=[177013385, 153674, 3] cpu-comparison-result=140 [max(70 (self) vs 140 (sum children))]
12:AGGREGATE
| output: count(cs_order_number), sum:merge(cs_ext_ship_cost), sum:merge(cs_net_profit)
| mem-estimate=16.00KB mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=9 row-size=40B cardinality=1 cost=4869719
+| tuple-ids=9 row-size=40B cardinality=1 cost=153674
| in pipelines: 12(GETNEXT), 11(OPEN)
|
11:AGGREGATE
| output: sum(cs_ext_ship_cost), sum(cs_net_profit)
| group by: cs_order_number
-| mem-estimate=38.33MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=8 row-size=40B cardinality=23.19M cost=68852014
+| mem-estimate=10.00MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
+| tuple-ids=8 row-size=40B cardinality=731.78K cost=2173116
| in pipelines: 11(GETNEXT), 05(OPEN)
|
10:HASH JOIN [RIGHT ANTI JOIN, PARTITIONED]
| hash-table-id=00
| hash predicates: cr1.cr_order_number = cs1.cs_order_number
-| mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,2,3,1 row-size=88B cardinality=23.19M cost=115014230
+| mem-estimate=0B mem-reservation=0B spill-buffer=512.00KB thread-reservation=0
+| tuple-ids=0,2,1,3 row-size=88B cardinality=731.78K cost=110944957
| in pipelines: 05(GETNEXT), 04(OPEN)
|
-|--F08:PLAN FRAGMENT [HASH(cs1.cs_order_number)] hosts=10 instances=30 (adjusted from 120)
-| | Per-Instance Resources: mem-estimate=101.98MB mem-reservation=34.00MB thread-reservation=1
-| | max-parallelism=30 segment-costs=[1853919983] cpu-comparison-result=150 [max(150 (self) vs 150 (sum children))]
+|--F08:PLAN FRAGMENT [HASH(cs1.cs_order_number)] hosts=10 instances=20 (adjusted from 120)
+| | Per-Instance Resources: mem-estimate=20.38MB mem-reservation=8.50MB thread-reservation=1
+| | max-parallelism=20 segment-costs=[1827393351] cpu-comparison-result=140 [max(140 (self) vs 140 (sum children))]
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: cs1.cs_order_number
-| | mem-estimate=90.10MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 cost=23189142
+| | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0 cost=731784
| |
| 09:HASH JOIN [RIGHT SEMI JOIN, PARTITIONED]
| | hash-table-id=01
| | hash predicates: cs2.cs_order_number = cs1.cs_order_number
| | other join predicates: cs1.cs_warehouse_sk != cs2.cs_warehouse_sk
-| | mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| | tuple-ids=0,2,3,1 row-size=88B cardinality=23.19M cost=1112301874
+| | mem-estimate=0B mem-reservation=0B spill-buffer=512.00KB thread-reservation=0
+| | tuple-ids=0,2,1,3 row-size=88B cardinality=731.78K cost=1108232600
| | in pipelines: 04(GETNEXT), 00(OPEN)
| |
-| |--F09:PLAN FRAGMENT [HASH(cs1.cs_order_number)] hosts=10 instances=30 (adjusted from 120)
-| | | Per-Instance Resources: mem-estimate=120.23MB mem-reservation=42.00MB thread-reservation=1 runtime-filters-memory=8.00MB
-| | | max-parallelism=30 segment-costs=[37659166] cpu-comparison-result=150 [max(150 (self) vs 42 (sum children))]
+| |--F09:PLAN FRAGMENT [HASH(cs1.cs_order_number)] hosts=10 instances=20 (adjusted from 120)
+| | | Per-Instance Resources: mem-estimate=27.74MB mem-reservation=9.50MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=20 segment-costs=[1188417] cpu-comparison-result=140 [max(140 (self) vs 42 (sum children))]
| | JOIN BUILD
| | | join-table-id=01 plan-id=02 cohort-id=02
| | | build expressions: cs1.cs_order_number
| | | runtime filters: RF000[bloom] <- cs1.cs_order_number, RF001[min_max] <- cs1.cs_order_number
-| | | mem-estimate=90.10MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 cost=23189142
+| | | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0 cost=731784
| | |
| | 18:EXCHANGE [HASH(cs1.cs_order_number)]
-| | | mem-estimate=22.13MB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=0,2,3,1 row-size=88B cardinality=23.19M cost=14470024
+| | | mem-estimate=18.24MB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=0,2,1,3 row-size=88B cardinality=731.78K cost=456633
| | | in pipelines: 00(GETNEXT)
| | |
| | F02:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
| | Per-Host Shared Resources: mem-estimate=3.00MB mem-reservation=3.00MB thread-reservation=0 runtime-filters-memory=3.00MB
-| | Per-Instance Resources: mem-estimate=28.13MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=230 segment-costs=[2238695495]
+| | Per-Instance Resources: mem-estimate=24.09MB mem-reservation=1.00MB thread-reservation=1
+| | max-parallelism=210 segment-costs=[2052747176]
| | 08:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash-table-id=02
-| | | hash predicates: cs1.cs_ship_date_sk = d_date_sk
-| | | fk/pk conjuncts: cs1.cs_ship_date_sk = d_date_sk
-| | | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=0,2,3,1 row-size=88B cardinality=23.19M cost=10149887
-| | | in pipelines: 00(GETNEXT), 01(OPEN)
-| | |
-| | |--F10:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | | max-parallelism=10 segment-costs=[17005]
-| | | JOIN BUILD
-| | | | join-table-id=02 plan-id=03 cohort-id=03
-| | | | build expressions: d_date_sk
-| | | | runtime filters: RF002[bloom] <- d_date_sk, RF003[min_max] <- d_date_sk
-| | | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
-| | | |
-| | | 16:EXCHANGE [BROADCAST]
-| | | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | | tuple-ids=1 row-size=8B cardinality=7.30K cost=9700
-| | | | in pipelines: 01(GETNEXT)
-| | | |
-| | | F05:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | | max-parallelism=1 segment-costs=[12835]
-| | | 01:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| | | HDFS partitions=1/1 files=1 size=2.17MB
-| | | predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
-| | | stored statistics:
-| | | table: rows=73.05K size=2.17MB
-| | | columns: all
-| | | extrapolated-rows=disabled max-scan-range-rows=73.05K
-| | | parquet statistics predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
-| | | parquet dictionary predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
-| | | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | | tuple-ids=1 row-size=8B cardinality=7.30K cost=12520
-| | | in pipelines: 01(GETNEXT)
-| | |
-| | 07:HASH JOIN [INNER JOIN, BROADCAST]
-| | | hash-table-id=03
| | | hash predicates: cs1.cs_call_center_sk = cc_call_center_sk
| | | fk/pk conjuncts: cs1.cs_call_center_sk = cc_call_center_sk
| | | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=0,2,3 row-size=80B cardinality=23.19M cost=24595065
+| | | tuple-ids=0,2,1,3 row-size=88B cardinality=731.78K cost=776150
| | | in pipelines: 00(GETNEXT), 03(OPEN)
| | |
-| | |--F11:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
+| | |--F10:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
| | | | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
| | | | max-parallelism=10 segment-costs=[24]
| | | JOIN BUILD
-| | | | join-table-id=03 plan-id=04 cohort-id=03
+| | | | join-table-id=02 plan-id=03 cohort-id=03
| | | | build expressions: cc_call_center_sk
-| | | | runtime filters: RF004[bloom] <- cc_call_center_sk, RF005[min_max] <- cc_call_center_sk
+| | | | runtime filters: RF002[bloom] <- cc_call_center_sk, RF003[min_max] <- cc_call_center_sk
| | | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=14
| | | |
-| | | 15:EXCHANGE [BROADCAST]
+| | | 16:EXCHANGE [BROADCAST]
| | | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
| | | | tuple-ids=3 row-size=30B cardinality=14 cost=10
| | | | in pipelines: 03(GETNEXT)
| | | |
-| | | F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | | F05:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| | | Per-Instance Resources: mem-estimate=16.13MB mem-reservation=16.00KB thread-reservation=1
| | | max-parallelism=1 segment-costs=[14]
| | | 03:SCAN HDFS [tpcds_partitioned_parquet_snap.call_center, RANDOM]
@@ -579,6 +541,44 @@ max-parallelism=30 segment-costs=[247761556, 4869719, 3] cpu-comparison-result=1
| | | tuple-ids=3 row-size=30B cardinality=14 cost=13
| | | in pipelines: 03(GETNEXT)
| | |
+| | 07:HASH JOIN [INNER JOIN, BROADCAST]
+| | | hash-table-id=03
+| | | hash predicates: cs1.cs_ship_date_sk = d_date_sk
+| | | fk/pk conjuncts: cs1.cs_ship_date_sk = d_date_sk
+| | | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| | | tuple-ids=0,2,1 row-size=58B cardinality=2.51M cost=20847819
+| | | in pipelines: 00(GETNEXT), 01(OPEN)
+| | |
+| | |--F11:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
+| | | | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | | max-parallelism=10 segment-costs=[141]
+| | | JOIN BUILD
+| | | | join-table-id=03 plan-id=04 cohort-id=03
+| | | | build expressions: d_date_sk
+| | | | runtime filters: RF004[bloom] <- d_date_sk, RF005[min_max] <- d_date_sk
+| | | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=61
+| | | |
+| | | 15:EXCHANGE [BROADCAST]
+| | | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | | tuple-ids=1 row-size=8B cardinality=61 cost=80
+| | | | in pipelines: 01(GETNEXT)
+| | | |
+| | | F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| | | max-parallelism=1 segment-costs=[12522]
+| | | 01:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| | | HDFS partitions=1/1 files=1 size=2.17MB
+| | | predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
+| | | stored statistics:
+| | | table: rows=73.05K size=2.17MB
+| | | columns: all
+| | | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | | parquet statistics predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
+| | | parquet dictionary predicates: d_date <= DATE '2001-04-02', d_date >= DATE '2001-02-01'
+| | | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | | tuple-ids=1 row-size=8B cardinality=61 cost=12520
+| | | in pipelines: 01(GETNEXT)
+| | |
| | 06:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash-table-id=04
| | | hash predicates: cs1.cs_ship_addr_sk = ca_address_sk
@@ -619,7 +619,7 @@ max-parallelism=30 segment-costs=[247761556, 4869719, 3] cpu-comparison-result=1
| | |
| | 00:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales cs1, RANDOM]
| | HDFS partitions=1831/1831 files=1831 size=150.82MB
-| | runtime filters: RF003[min_max] -> cs1.cs_ship_date_sk, RF005[min_max] -> cs1.cs_call_center_sk, RF007[min_max] -> cs1.cs_ship_addr_sk, RF002[bloom] -> cs1.cs_ship_date_sk, RF004[bloom] -> cs1.cs_call_center_sk, RF006[bloom] -> cs1.cs_ship_addr_sk
+| | runtime filters: RF003[min_max] -> cs1.cs_call_center_sk, RF005[min_max] -> cs1.cs_ship_date_sk, RF007[min_max] -> cs1.cs_ship_addr_sk, RF002[bloom] -> cs1.cs_call_center_sk, RF004[bloom] -> cs1.cs_ship_date_sk, RF006[bloom] -> cs1.cs_ship_addr_sk
| | stored statistics:
| | table: rows=4.32G size=280.96GB
| | partitions: 1831/1831 rows=4.32G
@@ -635,8 +635,8 @@ max-parallelism=30 segment-costs=[247761556, 4869719, 3] cpu-comparison-result=1
| | in pipelines: 04(GETNEXT)
| |
| F01:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
-| Per-Host Shared Resources: mem-estimate=8.00MB mem-reservation=8.00MB thread-reservation=0 runtime-filters-memory=8.00MB
-| Per-Instance Resources: mem-estimate=17.88MB mem-reservation=1.00MB thread-reservation=1
+| Per-Host Shared Resources: mem-estimate=1.00MB mem-reservation=1.00MB thread-reservation=0 runtime-filters-memory=1.00MB
+| Per-Instance Resources: mem-estimate=17.25MB mem-reservation=1.00MB thread-reservation=1
| max-parallelism=640 segment-costs=[6310769910]
| 04:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales cs2, RANDOM]
| HDFS partitions=1831/1831 files=1831 size=150.82MB
@@ -656,7 +656,7 @@ max-parallelism=30 segment-costs=[247761556, 4869719, 3] cpu-comparison-result=1
| in pipelines: 05(GETNEXT)
|
F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=50 (adjusted from 120)
-Per-Instance Resources: mem-estimate=17.41MB mem-reservation=32.00KB thread-reservation=1
+Per-Instance Resources: mem-estimate=16.94MB mem-reservation=32.00KB thread-reservation=1
max-parallelism=50 segment-costs=[494918664]
05:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_returns cr1, RANDOM]
HDFS partitions=2060/2060 files=2060 size=30.47MB
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q20.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q20.test
index fba2de312..e666ab345 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q20.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q20.test
@@ -31,10 +31,10 @@ limit 100;
# end query 20 in stream 0 using template query20.tpl
---- PLAN
Max Per-Host Resource Reservation: Memory=76.94MB Threads=1
-Per-Host Resource Estimates: Memory=255.55GB
+Per-Host Resource Estimates: Memory=4.79GB
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=255.55GB mem-reservation=76.94MB thread-reservation=1 runtime-filters-memory=2.00MB
-| max-parallelism=1 segment-costs=[9527420857, 11700907663, 23403078654, 700]
+| Per-Instance Resources: mem-estimate=4.79GB mem-reservation=76.94MB thread-reservation=1 runtime-filters-memory=2.00MB
+| max-parallelism=1 segment-costs=[197688508, 198103846, 323470699, 700]
PLAN-ROOT SINK
| output exprs: i_item_id, i_item_desc, i_category, i_class, i_current_price, sum(cs_ext_sales_price), sum(cs_ext_sales_price) * CAST(100 AS DECIMAL(3,0)) / sum(sum(cs_ext_sales_price))
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=700
@@ -42,56 +42,35 @@ PLAN-ROOT SINK
08:TOP-N [LIMIT=100]
| order by: i_category ASC, i_class ASC, i_item_id ASC, i_item_desc ASC, sum(cs_ext_sales_price) * 100 / sum(sum(cs_ext_sales_price)) ASC
| mem-estimate=20.93KB mem-reservation=0B thread-reservation=0
-| tuple-ids=5 row-size=214B cardinality=100 cost=22107702910
+| tuple-ids=5 row-size=214B cardinality=100 cost=301539159
| in pipelines: 08(GETNEXT), 06(OPEN)
|
07:ANALYTIC
| functions: sum(sum(cs_ext_sales_price))
| partition by: i_class
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=10,9 row-size=214B cardinality=1.30G cost=1295375744
+| tuple-ids=10,9 row-size=214B cardinality=21.93M cost=21931540
| in pipelines: 06(GETNEXT)
|
06:SORT
| order by: i_class ASC NULLS LAST
-| mem-estimate=239.22GB mem-reservation=12.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=10 row-size=198B cardinality=1.30G cost=11700907663
+| mem-estimate=4.05GB mem-reservation=12.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=10 row-size=198B cardinality=21.93M cost=198103846
| in pipelines: 06(GETNEXT), 05(OPEN)
|
05:AGGREGATE [FINALIZE]
| output: sum(cs_ext_sales_price)
| group by: i_item_id, i_item_desc, i_category, i_class, i_current_price
-| mem-estimate=16.32GB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=1.30G cost=7646084642
+| mem-estimate=754.90MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=21.93M cost=129453102
| in pipelines: 05(GETNEXT), 00(OPEN)
|
04:HASH JOIN [INNER JOIN]
-| hash predicates: cs_sold_date_sk = d_date_sk
-| fk/pk conjuncts: cs_sold_date_sk = d_date_sk
-| runtime filters: RF000[bloom] <- d_date_sk
-| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1,2 row-size=206B cardinality=1.30G cost=566993251
-| in pipelines: 00(GETNEXT), 02(OPEN)
-|
-|--02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
-| HDFS partitions=1/1 files=1 size=2.17MB
-| predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
-| stored statistics:
-| table: rows=73.05K size=2.17MB
-| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=73.05K
-| parquet statistics predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
-| parquet dictionary predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
-| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
-| in pipelines: 02(GETNEXT)
-|
-03:HASH JOIN [INNER JOIN]
| hash predicates: cs_item_sk = i_item_sk
| fk/pk conjuncts: cs_item_sk = i_item_sk
-| runtime filters: RF002[bloom] <- i_item_sk, RF003[min_max] <- i_item_sk
+| runtime filters: RF000[bloom] <- i_item_sk, RF001[min_max] <- i_item_sk
| mem-estimate=34.00MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,1 row-size=198B cardinality=1.30G cost=567093946
+| tuple-ids=0,2,1 row-size=206B cardinality=21.93M cost=22842836
| in pipelines: 00(GETNEXT), 01(OPEN)
|
|--01:SCAN HDFS [tpcds_partitioned_parquet_snap.item]
@@ -107,23 +86,44 @@ PLAN-ROOT SINK
| tuple-ids=1 row-size=186B cardinality=108.00K cost=727024
| in pipelines: 01(GETNEXT)
|
+03:HASH JOIN [INNER JOIN]
+| hash predicates: cs_sold_date_sk = d_date_sk
+| fk/pk conjuncts: cs_sold_date_sk = d_date_sk
+| runtime filters: RF002[bloom] <- d_date_sk
+| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,2 row-size=20B cardinality=73.14M cost=32014144
+| in pipelines: 00(GETNEXT), 02(OPEN)
+|
+|--02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
+| HDFS partitions=1/1 files=1 size=2.17MB
+| predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
+| stored statistics:
+| table: rows=73.05K size=2.17MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=73.05K
+| parquet statistics predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
+| parquet dictionary predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
+| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| tuple-ids=2 row-size=8B cardinality=31 cost=12520
+| in pipelines: 02(GETNEXT)
+|
00:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales]
HDFS partitions=1831/1831 files=1831 size=150.82MB
- runtime filters: RF003[min_max] -> cs_item_sk, RF000[bloom] -> cs_sold_date_sk, RF002[bloom] -> cs_item_sk
+ runtime filters: RF001[min_max] -> cs_item_sk, RF000[bloom] -> cs_item_sk, RF002[bloom] -> cs_sold_date_sk
stored statistics:
table: rows=4.32G size=280.96GB
partitions: 1831/1831 rows=4.32G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=21.52M
+ extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=31(filtered from 1831)
mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
- tuple-ids=0 row-size=12B cardinality=1.30G(filtered from 4.32G) cost=746509474
+ tuple-ids=0 row-size=12B cardinality=73.14M(filtered from 4.32G) cost=12638882
in pipelines: 00(GETNEXT)
---- DISTRIBUTEDPLAN
-Max Per-Host Resource Reservation: Memory=1.43GB Threads=41
-Per-Host Resource Estimates: Memory=41.27GB
+Max Per-Host Resource Reservation: Memory=388.31MB Threads=15
+Per-Host Resource Estimates: Memory=1.60GB
F05:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=6.50MB mem-reservation=4.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[768] cpu-comparison-result=120 [max(1 (self) vs 120 (sum children))]
+| Per-Instance Resources: mem-estimate=4.83MB mem-reservation=4.00MB thread-reservation=1
+| max-parallelism=1 segment-costs=[768] cpu-comparison-result=40 [max(1 (self) vs 40 (sum children))]
PLAN-ROOT SINK
| output exprs: i_item_id, i_item_desc, i_category, i_class, i_current_price, sum(cs_ext_sales_price), sum(cs_ext_sales_price) * CAST(100 AS DECIMAL(3,0)) / sum(sum(cs_ext_sales_price))
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=700
@@ -131,124 +131,86 @@ PLAN-ROOT SINK
14:MERGING-EXCHANGE [UNPARTITIONED]
| order by: i_category ASC, i_class ASC, i_item_id ASC, i_item_desc ASC, sum(cs_ext_sales_price) * 100 / sum(sum(cs_ext_sales_price)) ASC
| limit: 100
-| mem-estimate=2.50MB mem-reservation=0B thread-reservation=0
+| mem-estimate=854.83KB mem-reservation=0B thread-reservation=0
| tuple-ids=5 row-size=214B cardinality=100 cost=68
| in pipelines: 08(GETNEXT)
|
-F04:PLAN FRAGMENT [HASH(i_class)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=2.03GB mem-reservation=16.00MB thread-reservation=1
-max-parallelism=2350 segment-costs=[13026395726, 23403078654, 592] cpu-comparison-result=120 [max(120 (self) vs 120 (sum children))]
+F04:PLAN FRAGMENT [HASH(i_class)] hosts=10 instances=40 (adjusted from 120)
+Per-Instance Resources: mem-estimate=123.61MB mem-reservation=16.00MB thread-reservation=1
+max-parallelism=40 segment-costs=[220545206, 323470699, 592] cpu-comparison-result=40 [max(40 (self) vs 30 (sum children))]
08:TOP-N [LIMIT=100]
| order by: i_category ASC, i_class ASC, i_item_id ASC, i_item_desc ASC, sum(cs_ext_sales_price) * 100 / sum(sum(cs_ext_sales_price)) ASC
| mem-estimate=20.93KB mem-reservation=0B thread-reservation=0
-| tuple-ids=5 row-size=214B cardinality=100 cost=22107702910
+| tuple-ids=5 row-size=214B cardinality=100 cost=301539159
| in pipelines: 08(GETNEXT), 06(OPEN)
|
07:ANALYTIC
| functions: sum(sum(cs_ext_sales_price))
| partition by: i_class
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=10,9 row-size=214B cardinality=1.30G cost=1295375744
+| tuple-ids=10,9 row-size=214B cardinality=21.93M cost=21931540
| in pipelines: 06(GETNEXT)
|
06:SORT
| order by: i_class ASC NULLS LAST
-| mem-estimate=1.99GB mem-reservation=12.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=10 row-size=198B cardinality=1.30G cost=11700907663
+| mem-estimate=103.68MB mem-reservation=12.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=10 row-size=198B cardinality=21.93M cost=198103846
| in pipelines: 06(GETNEXT), 12(OPEN)
|
13:EXCHANGE [HASH(i_class)]
-| mem-estimate=33.71MB mem-reservation=0B thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=1.30G cost=1325488063
+| mem-estimate=15.93MB mem-reservation=0B thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=21.93M cost=22441360
| in pipelines: 12(GETNEXT)
|
-F03:PLAN FRAGMENT [HASH(i_item_id,i_item_desc,i_category,i_class,i_current_price)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=710.13MB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=1730 segment-costs=[8971572705, 17209383085] cpu-comparison-result=120 [max(120 (self) vs 120 (sum children))]
+F03:PLAN FRAGMENT [HASH(i_item_id,i_item_desc,i_category,i_class,i_current_price)] hosts=10 instances=30 (adjusted from 120)
+Per-Instance Resources: mem-estimate=178.22MB mem-reservation=34.00MB thread-reservation=1
+max-parallelism=30 segment-costs=[151894462, 291365859] cpu-comparison-result=30 [max(30 (self) vs 30 (sum children))]
12:AGGREGATE [FINALIZE]
| output: sum:merge(cs_ext_sales_price)
| group by: i_item_id, i_item_desc, i_category, i_class, i_current_price
-| mem-estimate=615.31MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=1.30G cost=7646084642
+| mem-estimate=146.61MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=21.93M cost=129453102
| in pipelines: 12(GETNEXT), 00(OPEN)
|
11:EXCHANGE [HASH(i_item_id,i_item_desc,i_category,i_class,i_current_price)]
-| mem-estimate=33.71MB mem-reservation=0B thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=1.30G cost=1325488063
+| mem-estimate=15.93MB mem-reservation=0B thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=21.93M cost=22441360
| in pipelines: 00(GETNEXT)
|
-F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=30 (adjusted from 120)
Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
-Per-Instance Resources: mem-estimate=692.30MB mem-reservation=35.00MB thread-reservation=1
-max-parallelism=1730 segment-costs=[9526566008, 17209383085] cpu-comparison-result=120 [max(120 (self) vs 25 (sum children))]
+Per-Instance Resources: mem-estimate=149.67MB mem-reservation=35.00MB thread-reservation=1
+max-parallelism=30 segment-costs=[196840933, 291365859] cpu-comparison-result=30 [max(30 (self) vs 25 (sum children))]
05:AGGREGATE [STREAMING]
| output: sum(cs_ext_sales_price)
| group by: i_item_id, i_item_desc, i_category, i_class, i_current_price
-| mem-estimate=581.48MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=1.30G cost=7646084642
+| mem-estimate=109.96MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=21.93M cost=129453102
| in pipelines: 00(GETNEXT)
|
04:HASH JOIN [INNER JOIN, BROADCAST]
| hash-table-id=00
-| hash predicates: cs_sold_date_sk = d_date_sk
-| fk/pk conjuncts: cs_sold_date_sk = d_date_sk
-| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1,2 row-size=206B cardinality=1.30G cost=566985946
-| in pipelines: 00(GETNEXT), 02(OPEN)
-|
-|--F06:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
-| JOIN BUILD
-| | join-table-id=00 plan-id=01 cohort-id=01
-| | build expressions: d_date_sk
-| | runtime filters: RF000[bloom] <- d_date_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
-| |
-| 10:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=2 row-size=8B cardinality=7.30K cost=9700
-| | in pipelines: 02(GETNEXT)
-| |
-| F02:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
-| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| HDFS partitions=1/1 files=1 size=2.17MB
-| predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
-| stored statistics:
-| table: rows=73.05K size=2.17MB
-| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=73.05K
-| parquet statistics predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
-| parquet dictionary predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
-| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
-| in pipelines: 02(GETNEXT)
-|
-03:HASH JOIN [INNER JOIN, BROADCAST]
-| hash-table-id=01
| hash predicates: cs_item_sk = i_item_sk
| fk/pk conjuncts: cs_item_sk = i_item_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,1 row-size=198B cardinality=1.30G cost=566985946
+| tuple-ids=0,2,1 row-size=206B cardinality=21.93M cost=22734836
| in pipelines: 00(GETNEXT), 01(OPEN)
|
-|--F07:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=419.74MB mem-reservation=409.00MB thread-reservation=1 runtime-filters-memory=1.00MB
+|--F06:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
+| | Per-Instance Resources: mem-estimate=113.74MB mem-reservation=103.00MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[251530]
| JOIN BUILD
-| | join-table-id=01 plan-id=02 cohort-id=01
+| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: i_item_sk
-| | runtime filters: RF002[bloom] <- i_item_sk, RF003[min_max] <- i_item_sk
-| | mem-estimate=408.00MB mem-reservation=408.00MB spill-buffer=2.00MB thread-reservation=0 cost=108000
+| | runtime filters: RF000[bloom] <- i_item_sk, RF001[min_max] <- i_item_sk
+| | mem-estimate=102.00MB mem-reservation=102.00MB spill-buffer=2.00MB thread-reservation=0 cost=108000
| |
-| 09:EXCHANGE [BROADCAST]
+| 10:EXCHANGE [BROADCAST]
| | mem-estimate=10.74MB mem-reservation=0B thread-reservation=0
| | tuple-ids=1 row-size=186B cardinality=108.00K cost=143530
| | in pipelines: 01(GETNEXT)
| |
-| F01:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
+| F02:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
| Per-Instance Resources: mem-estimate=16.74MB mem-reservation=512.00KB thread-reservation=1
| max-parallelism=4 segment-costs=[783679]
| 01:SCAN HDFS [tpcds_partitioned_parquet_snap.item, RANDOM]
@@ -264,23 +226,61 @@ max-parallelism=1730 segment-costs=[9526566008, 17209383085] cpu-comparison-resu
| tuple-ids=1 row-size=186B cardinality=108.00K cost=727024
| in pipelines: 01(GETNEXT)
|
+03:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=01
+| hash predicates: cs_sold_date_sk = d_date_sk
+| fk/pk conjuncts: cs_sold_date_sk = d_date_sk
+| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,2 row-size=20B cardinality=73.14M cost=32014113
+| in pipelines: 00(GETNEXT), 02(OPEN)
+|
+|--F07:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
+| | Per-Instance Resources: mem-estimate=6.83MB mem-reservation=6.81MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[71]
+| JOIN BUILD
+| | join-table-id=01 plan-id=02 cohort-id=01
+| | build expressions: d_date_sk
+| | runtime filters: RF002[bloom] <- d_date_sk
+| | mem-estimate=5.81MB mem-reservation=5.81MB spill-buffer=64.00KB thread-reservation=0 cost=31
+| |
+| 09:EXCHANGE [BROADCAST]
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=2 row-size=8B cardinality=31 cost=40
+| | in pipelines: 02(GETNEXT)
+| |
+| F01:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| max-parallelism=1 segment-costs=[12521]
+| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| HDFS partitions=1/1 files=1 size=2.17MB
+| predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
+| stored statistics:
+| table: rows=73.05K size=2.17MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=73.05K
+| parquet statistics predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
+| parquet dictionary predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
+| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| tuple-ids=2 row-size=8B cardinality=31 cost=12520
+| in pipelines: 02(GETNEXT)
+|
00:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales, RANDOM]
HDFS partitions=1831/1831 files=1831 size=150.82MB
- runtime filters: RF003[min_max] -> cs_item_sk, RF000[bloom] -> cs_sold_date_sk, RF002[bloom] -> cs_item_sk
+ runtime filters: RF001[min_max] -> cs_item_sk, RF000[bloom] -> cs_item_sk, RF002[bloom] -> cs_sold_date_sk
stored statistics:
table: rows=4.32G size=280.96GB
partitions: 1831/1831 rows=4.32G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=21.52M
+ extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=31(filtered from 1831)
mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
- tuple-ids=0 row-size=12B cardinality=1.30G(filtered from 4.32G) cost=746509474
+ tuple-ids=0 row-size=12B cardinality=73.14M(filtered from 4.32G) cost=12638882
in pipelines: 00(GETNEXT)
---- PARALLELPLANS
-Max Per-Host Resource Reservation: Memory=1.43GB Threads=41
-Per-Host Resource Estimates: Memory=41.27GB
+Max Per-Host Resource Reservation: Memory=388.31MB Threads=15
+Per-Host Resource Estimates: Memory=1.60GB
F05:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=6.50MB mem-reservation=4.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[768] cpu-comparison-result=120 [max(1 (self) vs 120 (sum children))]
+| Per-Instance Resources: mem-estimate=4.83MB mem-reservation=4.00MB thread-reservation=1
+| max-parallelism=1 segment-costs=[768] cpu-comparison-result=40 [max(1 (self) vs 40 (sum children))]
PLAN-ROOT SINK
| output exprs: i_item_id, i_item_desc, i_category, i_class, i_current_price, sum(cs_ext_sales_price), sum(cs_ext_sales_price) * CAST(100 AS DECIMAL(3,0)) / sum(sum(cs_ext_sales_price))
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=700
@@ -288,124 +288,86 @@ PLAN-ROOT SINK
14:MERGING-EXCHANGE [UNPARTITIONED]
| order by: i_category ASC, i_class ASC, i_item_id ASC, i_item_desc ASC, sum(cs_ext_sales_price) * 100 / sum(sum(cs_ext_sales_price)) ASC
| limit: 100
-| mem-estimate=2.50MB mem-reservation=0B thread-reservation=0
+| mem-estimate=854.83KB mem-reservation=0B thread-reservation=0
| tuple-ids=5 row-size=214B cardinality=100 cost=68
| in pipelines: 08(GETNEXT)
|
-F04:PLAN FRAGMENT [HASH(i_class)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=2.03GB mem-reservation=16.00MB thread-reservation=1
-max-parallelism=2350 segment-costs=[13026395726, 23403078654, 592] cpu-comparison-result=120 [max(120 (self) vs 120 (sum children))]
+F04:PLAN FRAGMENT [HASH(i_class)] hosts=10 instances=40 (adjusted from 120)
+Per-Instance Resources: mem-estimate=123.61MB mem-reservation=16.00MB thread-reservation=1
+max-parallelism=40 segment-costs=[220545206, 323470699, 592] cpu-comparison-result=40 [max(40 (self) vs 30 (sum children))]
08:TOP-N [LIMIT=100]
| order by: i_category ASC, i_class ASC, i_item_id ASC, i_item_desc ASC, sum(cs_ext_sales_price) * 100 / sum(sum(cs_ext_sales_price)) ASC
| mem-estimate=20.93KB mem-reservation=0B thread-reservation=0
-| tuple-ids=5 row-size=214B cardinality=100 cost=22107702910
+| tuple-ids=5 row-size=214B cardinality=100 cost=301539159
| in pipelines: 08(GETNEXT), 06(OPEN)
|
07:ANALYTIC
| functions: sum(sum(cs_ext_sales_price))
| partition by: i_class
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=10,9 row-size=214B cardinality=1.30G cost=1295375744
+| tuple-ids=10,9 row-size=214B cardinality=21.93M cost=21931540
| in pipelines: 06(GETNEXT)
|
06:SORT
| order by: i_class ASC NULLS LAST
-| mem-estimate=1.99GB mem-reservation=12.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=10 row-size=198B cardinality=1.30G cost=11700907663
+| mem-estimate=103.68MB mem-reservation=12.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=10 row-size=198B cardinality=21.93M cost=198103846
| in pipelines: 06(GETNEXT), 12(OPEN)
|
13:EXCHANGE [HASH(i_class)]
-| mem-estimate=33.71MB mem-reservation=0B thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=1.30G cost=1325488063
+| mem-estimate=15.93MB mem-reservation=0B thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=21.93M cost=22441360
| in pipelines: 12(GETNEXT)
|
-F03:PLAN FRAGMENT [HASH(i_item_id,i_item_desc,i_category,i_class,i_current_price)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=710.13MB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=1730 segment-costs=[8971572705, 17209383085] cpu-comparison-result=120 [max(120 (self) vs 120 (sum children))]
+F03:PLAN FRAGMENT [HASH(i_item_id,i_item_desc,i_category,i_class,i_current_price)] hosts=10 instances=30 (adjusted from 120)
+Per-Instance Resources: mem-estimate=178.22MB mem-reservation=34.00MB thread-reservation=1
+max-parallelism=30 segment-costs=[151894462, 291365859] cpu-comparison-result=30 [max(30 (self) vs 30 (sum children))]
12:AGGREGATE [FINALIZE]
| output: sum:merge(cs_ext_sales_price)
| group by: i_item_id, i_item_desc, i_category, i_class, i_current_price
-| mem-estimate=615.31MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=1.30G cost=7646084642
+| mem-estimate=146.61MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=21.93M cost=129453102
| in pipelines: 12(GETNEXT), 00(OPEN)
|
11:EXCHANGE [HASH(i_item_id,i_item_desc,i_category,i_class,i_current_price)]
-| mem-estimate=33.71MB mem-reservation=0B thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=1.30G cost=1325488063
+| mem-estimate=15.93MB mem-reservation=0B thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=21.93M cost=22441360
| in pipelines: 00(GETNEXT)
|
-F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=30 (adjusted from 120)
Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
-Per-Instance Resources: mem-estimate=692.30MB mem-reservation=35.00MB thread-reservation=1
-max-parallelism=1730 segment-costs=[9526566008, 17209383085] cpu-comparison-result=120 [max(120 (self) vs 25 (sum children))]
+Per-Instance Resources: mem-estimate=149.67MB mem-reservation=35.00MB thread-reservation=1
+max-parallelism=30 segment-costs=[196840933, 291365859] cpu-comparison-result=30 [max(30 (self) vs 25 (sum children))]
05:AGGREGATE [STREAMING]
| output: sum(cs_ext_sales_price)
| group by: i_item_id, i_item_desc, i_category, i_class, i_current_price
-| mem-estimate=581.48MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=3 row-size=198B cardinality=1.30G cost=7646084642
+| mem-estimate=109.96MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=3 row-size=198B cardinality=21.93M cost=129453102
| in pipelines: 00(GETNEXT)
|
04:HASH JOIN [INNER JOIN, BROADCAST]
| hash-table-id=00
-| hash predicates: cs_sold_date_sk = d_date_sk
-| fk/pk conjuncts: cs_sold_date_sk = d_date_sk
-| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1,2 row-size=206B cardinality=1.30G cost=566985946
-| in pipelines: 00(GETNEXT), 02(OPEN)
-|
-|--F06:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
-| JOIN BUILD
-| | join-table-id=00 plan-id=01 cohort-id=01
-| | build expressions: d_date_sk
-| | runtime filters: RF000[bloom] <- d_date_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
-| |
-| 10:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=2 row-size=8B cardinality=7.30K cost=9700
-| | in pipelines: 02(GETNEXT)
-| |
-| F02:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
-| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| HDFS partitions=1/1 files=1 size=2.17MB
-| predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
-| stored statistics:
-| table: rows=73.05K size=2.17MB
-| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=73.05K
-| parquet statistics predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
-| parquet dictionary predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
-| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
-| in pipelines: 02(GETNEXT)
-|
-03:HASH JOIN [INNER JOIN, BROADCAST]
-| hash-table-id=01
| hash predicates: cs_item_sk = i_item_sk
| fk/pk conjuncts: cs_item_sk = i_item_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,1 row-size=198B cardinality=1.30G cost=566985946
+| tuple-ids=0,2,1 row-size=206B cardinality=21.93M cost=22734836
| in pipelines: 00(GETNEXT), 01(OPEN)
|
-|--F07:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=419.74MB mem-reservation=409.00MB thread-reservation=1 runtime-filters-memory=1.00MB
+|--F06:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
+| | Per-Instance Resources: mem-estimate=113.74MB mem-reservation=103.00MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[251530]
| JOIN BUILD
-| | join-table-id=01 plan-id=02 cohort-id=01
+| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: i_item_sk
-| | runtime filters: RF002[bloom] <- i_item_sk, RF003[min_max] <- i_item_sk
-| | mem-estimate=408.00MB mem-reservation=408.00MB spill-buffer=2.00MB thread-reservation=0 cost=108000
+| | runtime filters: RF000[bloom] <- i_item_sk, RF001[min_max] <- i_item_sk
+| | mem-estimate=102.00MB mem-reservation=102.00MB spill-buffer=2.00MB thread-reservation=0 cost=108000
| |
-| 09:EXCHANGE [BROADCAST]
+| 10:EXCHANGE [BROADCAST]
| | mem-estimate=10.74MB mem-reservation=0B thread-reservation=0
| | tuple-ids=1 row-size=186B cardinality=108.00K cost=143530
| | in pipelines: 01(GETNEXT)
| |
-| F01:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
+| F02:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
| Per-Instance Resources: mem-estimate=16.74MB mem-reservation=512.00KB thread-reservation=1
| max-parallelism=4 segment-costs=[783679]
| 01:SCAN HDFS [tpcds_partitioned_parquet_snap.item, RANDOM]
@@ -421,15 +383,53 @@ max-parallelism=1730 segment-costs=[9526566008, 17209383085] cpu-comparison-resu
| tuple-ids=1 row-size=186B cardinality=108.00K cost=727024
| in pipelines: 01(GETNEXT)
|
+03:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=01
+| hash predicates: cs_sold_date_sk = d_date_sk
+| fk/pk conjuncts: cs_sold_date_sk = d_date_sk
+| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,2 row-size=20B cardinality=73.14M cost=32014113
+| in pipelines: 00(GETNEXT), 02(OPEN)
+|
+|--F07:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
+| | Per-Instance Resources: mem-estimate=6.83MB mem-reservation=6.81MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[71]
+| JOIN BUILD
+| | join-table-id=01 plan-id=02 cohort-id=01
+| | build expressions: d_date_sk
+| | runtime filters: RF002[bloom] <- d_date_sk
+| | mem-estimate=5.81MB mem-reservation=5.81MB spill-buffer=64.00KB thread-reservation=0 cost=31
+| |
+| 09:EXCHANGE [BROADCAST]
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=2 row-size=8B cardinality=31 cost=40
+| | in pipelines: 02(GETNEXT)
+| |
+| F01:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| max-parallelism=1 segment-costs=[12521]
+| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| HDFS partitions=1/1 files=1 size=2.17MB
+| predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
+| stored statistics:
+| table: rows=73.05K size=2.17MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=73.05K
+| parquet statistics predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
+| parquet dictionary predicates: d_date <= DATE '1998-06-07', d_date >= DATE '1998-05-08'
+| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| tuple-ids=2 row-size=8B cardinality=31 cost=12520
+| in pipelines: 02(GETNEXT)
+|
00:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales, RANDOM]
HDFS partitions=1831/1831 files=1831 size=150.82MB
- runtime filters: RF003[min_max] -> cs_item_sk, RF000[bloom] -> cs_sold_date_sk, RF002[bloom] -> cs_item_sk
+ runtime filters: RF001[min_max] -> cs_item_sk, RF000[bloom] -> cs_item_sk, RF002[bloom] -> cs_sold_date_sk
stored statistics:
table: rows=4.32G size=280.96GB
partitions: 1831/1831 rows=4.32G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=21.52M
+ extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=31(filtered from 1831)
mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
- tuple-ids=0 row-size=12B cardinality=1.30G(filtered from 4.32G) cost=746509474
+ tuple-ids=0 row-size=12B cardinality=73.14M(filtered from 4.32G) cost=12638882
in pipelines: 00(GETNEXT)
====
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q21.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q21.test
index f0788a3d0..0bcc7c23c 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q21.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q21.test
@@ -34,7 +34,7 @@ Max Per-Host Resource Reservation: Memory=44.00MB Threads=1
Per-Host Resource Estimates: Memory=338MB
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=338.11MB mem-reservation=44.00MB thread-reservation=1 runtime-filters-memory=3.00MB
-| max-parallelism=1 segment-costs=[540810996, 4080496, 400]
+| max-parallelism=1 segment-costs=[154409711, 4080496, 400]
PLAN-ROOT SINK
| output exprs: w_warehouse_name, i_item_id, inv_before, inv_after
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=400
@@ -50,15 +50,33 @@ PLAN-ROOT SINK
| group by: w_warehouse_name, i_item_id
| having: (CASE WHEN sum(CASE WHEN d_date < DATE '2000-05-22' THEN inv_quantity_on_hand ELSE 0 END) > CAST(0 AS BIGINT) THEN CAST(sum(CASE WHEN d_date >= DATE '2000-05-22' THEN inv_quantity_on_hand ELSE 0 END) AS DOUBLE) / CAST(sum(CASE WHEN d_date < DATE '2000-05-22' THEN inv_quantity_on_hand ELSE 0 END) AS DOUBLE) ELSE NULL END) <= CAST(1.5 AS DOUBLE), (CASE WHEN sum(CASE WHEN d_date < DATE '2000-05-22' THEN inv_quantity_on_hand ELSE 0 END) > CAST(0 AS BIGINT) THEN CAST(sum(CASE WHEN [...]
| mem-estimate=312.36MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=4 row-size=72B cardinality=389.70K cost=166829332
+| tuple-ids=4 row-size=72B cardinality=389.70K cost=52513771
| in pipelines: 07(GETNEXT), 00(OPEN)
|
06:HASH JOIN [INNER JOIN]
+| hash predicates: inv_warehouse_sk = w_warehouse_sk
+| fk/pk conjuncts: inv_warehouse_sk = w_warehouse_sk
+| runtime filters: RF000[bloom] <- w_warehouse_sk, RF001[min_max] <- w_warehouse_sk
+| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,2,3,1 row-size=92B cardinality=24.14M cost=10567819
+| in pipelines: 00(GETNEXT), 01(OPEN)
+|
+|--01:SCAN HDFS [tpcds_partitioned_parquet_snap.warehouse]
+| HDFS partitions=1/1 files=1 size=4.38KB
+| stored statistics:
+| table: rows=22 size=5.99KB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=22
+| mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=0
+| tuple-ids=1 row-size=32B cardinality=22 cost=6
+| in pipelines: 01(GETNEXT)
+|
+05:HASH JOIN [INNER JOIN]
| hash predicates: inv_date_sk = d_date_sk
| fk/pk conjuncts: inv_date_sk = d_date_sk
-| runtime filters: RF000[bloom] <- d_date_sk
+| runtime filters: RF002[bloom] <- d_date_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,2,1,3 row-size=92B cardinality=103.30M cost=45223618
+| tuple-ids=0,2,3 row-size=60B cardinality=24.14M cost=10567858
| in pipelines: 00(GETNEXT), 03(OPEN)
|
|--03:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
@@ -71,33 +89,15 @@ PLAN-ROOT SINK
| parquet statistics predicates: d_date <= DATE '2000-06-21', d_date >= DATE '2000-04-22'
| parquet dictionary predicates: d_date <= DATE '2000-06-21', d_date >= DATE '2000-04-22'
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=3 row-size=8B cardinality=7.30K cost=12520
+| tuple-ids=3 row-size=8B cardinality=61 cost=12520
| in pipelines: 03(GETNEXT)
|
-05:HASH JOIN [INNER JOIN]
-| hash predicates: inv_warehouse_sk = w_warehouse_sk
-| fk/pk conjuncts: inv_warehouse_sk = w_warehouse_sk
-| runtime filters: RF002[bloom] <- w_warehouse_sk, RF003[min_max] <- w_warehouse_sk
-| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,2,1 row-size=84B cardinality=103.30M cost=45216335
-| in pipelines: 00(GETNEXT), 01(OPEN)
-|
-|--01:SCAN HDFS [tpcds_partitioned_parquet_snap.warehouse]
-| HDFS partitions=1/1 files=1 size=4.38KB
-| stored statistics:
-| table: rows=22 size=5.99KB
-| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=22
-| mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=0
-| tuple-ids=1 row-size=32B cardinality=22 cost=6
-| in pipelines: 01(GETNEXT)
-|
04:HASH JOIN [INNER JOIN]
| hash predicates: inv_item_sk = i_item_sk
| fk/pk conjuncts: inv_item_sk = i_item_sk
| runtime filters: RF004[bloom] <- i_item_sk, RF005[min_max] <- i_item_sk
| mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
-| tuple-ids=0,2 row-size=52B cardinality=103.30M cost=45252313
+| tuple-ids=0,2 row-size=52B cardinality=24.14M(filtered from 103.30M) cost=24947665
| in pipelines: 00(GETNEXT), 02(OPEN)
|
|--02:SCAN HDFS [tpcds_partitioned_parquet_snap.item]
@@ -115,21 +115,21 @@ PLAN-ROOT SINK
|
00:SCAN HDFS [tpcds_partitioned_parquet_snap.inventory]
HDFS partitions=261/261 files=261 size=44.10MB
- runtime filters: RF003[min_max] -> inv_warehouse_sk, RF005[min_max] -> inv_item_sk, RF000[bloom] -> inv_date_sk, RF002[bloom] -> inv_warehouse_sk, RF004[bloom] -> inv_item_sk
+ runtime filters: RF001[min_max] -> inv_warehouse_sk, RF005[min_max] -> inv_item_sk, RF000[bloom] -> inv_warehouse_sk, RF002[bloom] -> inv_date_sk, RF004[bloom] -> inv_item_sk
stored statistics:
table: rows=1.03G size=5.10GB
partitions: 261/261 rows=1.03G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=3.98M
+ extrapolated-rows=disabled max-scan-range-rows=3.98M est-scan-range=61(filtered from 261)
mem-estimate=16.00MB mem-reservation=256.00KB thread-reservation=0
- tuple-ids=0 row-size=16B cardinality=103.30M(filtered from 1.03G) cost=238132224
+ tuple-ids=0 row-size=16B cardinality=24.14M(filtered from 1.03G) cost=55655424
in pipelines: 00(GETNEXT)
---- DISTRIBUTEDPLAN
-Max Per-Host Resource Reservation: Memory=455.52MB Threads=18
-Per-Host Resource Estimates: Memory=990MB
+Max Per-Host Resource Reservation: Memory=168.52MB Threads=11
+Per-Host Resource Estimates: Memory=328MB
F05:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=4.07MB mem-reservation=4.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[439] cpu-comparison-result=100 [max(1 (self) vs 100 (sum children))]
+| max-parallelism=1 segment-costs=[439] cpu-comparison-result=36 [max(1 (self) vs 36 (sum children))]
PLAN-ROOT SINK
| output exprs: w_warehouse_name, i_item_id, inv_before, inv_after
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=400
@@ -142,8 +142,8 @@ PLAN-ROOT SINK
| in pipelines: 08(GETNEXT)
|
F04:PLAN FRAGMENT [HASH(w_warehouse_name,i_item_id)] hosts=10 instances=10 (adjusted from 120)
-Per-Instance Resources: mem-estimate=51.43MB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=10 segment-costs=[24999660, 4080496, 220] cpu-comparison-result=100 [max(10 (self) vs 100 (sum children))]
+Per-Instance Resources: mem-estimate=46.23MB mem-reservation=34.00MB thread-reservation=1
+max-parallelism=10 segment-costs=[24999660, 4080496, 220] cpu-comparison-result=36 [max(10 (self) vs 36 (sum children))]
08:TOP-N [LIMIT=100]
| order by: w_warehouse_name ASC, i_item_id ASC
| mem-estimate=7.04KB mem-reservation=0B thread-reservation=0
@@ -159,110 +159,110 @@ max-parallelism=10 segment-costs=[24999660, 4080496, 220] cpu-comparison-result=
| in pipelines: 13(GETNEXT), 00(OPEN)
|
12:EXCHANGE [HASH(w_warehouse_name,i_item_id)]
-| mem-estimate=17.43MB mem-reservation=0B thread-reservation=0
+| mem-estimate=12.23MB mem-reservation=0B thread-reservation=0
| tuple-ids=4 row-size=72B cardinality=3.90M cost=1724492
| in pipelines: 00(GETNEXT)
|
-F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=100 (adjusted from 120)
+F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=30 (adjusted from 120)
Per-Host Shared Resources: mem-estimate=3.00MB mem-reservation=3.00MB thread-reservation=0 runtime-filters-memory=3.00MB
-Per-Instance Resources: mem-estimate=81.07MB mem-reservation=34.25MB thread-reservation=1
-max-parallelism=100 segment-costs=[990776711, 20089249] cpu-comparison-result=100 [max(100 (self) vs 36 (sum children))]
+Per-Instance Resources: mem-estimate=67.35MB mem-reservation=34.25MB thread-reservation=1
+max-parallelism=30 segment-costs=[245904713, 20089249] cpu-comparison-result=36 [max(30 (self) vs 36 (sum children))]
07:AGGREGATE [STREAMING]
| output: sum(CAST(CASE WHEN d_date < DATE '2000-05-22' THEN inv_quantity_on_hand ELSE CAST(0 AS INT) END AS BIGINT)), sum(CAST(CASE WHEN d_date >= DATE '2000-05-22' THEN inv_quantity_on_hand ELSE CAST(0 AS INT) END AS BIGINT))
| group by: w_warehouse_name, i_item_id
-| mem-estimate=62.10MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=4 row-size=72B cardinality=3.90M cost=616995548
+| mem-estimate=48.38MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=4 row-size=72B cardinality=3.90M cost=144202030
| in pipelines: 00(GETNEXT)
|
06:HASH JOIN [INNER JOIN, BROADCAST]
| hash-table-id=00
-| hash predicates: inv_date_sk = d_date_sk
-| fk/pk conjuncts: inv_date_sk = d_date_sk
+| hash predicates: inv_warehouse_sk = w_warehouse_sk
+| fk/pk conjuncts: inv_warehouse_sk = w_warehouse_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,2,1,3 row-size=92B cardinality=103.30M cost=45216313
-| in pipelines: 00(GETNEXT), 03(OPEN)
+| tuple-ids=0,2,3,1 row-size=92B cardinality=24.14M cost=10567797
+| in pipelines: 00(GETNEXT), 01(OPEN)
|
|--F06:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=20.44MB mem-reservation=20.38MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
+| | Per-Instance Resources: mem-estimate=6.83MB mem-reservation=6.81MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[42]
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
-| | build expressions: d_date_sk
-| | runtime filters: RF000[bloom] <- d_date_sk
-| | mem-estimate=19.38MB mem-reservation=19.38MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | build expressions: w_warehouse_sk
+| | runtime filters: RF000[bloom] <- w_warehouse_sk, RF001[min_max] <- w_warehouse_sk
+| | mem-estimate=5.81MB mem-reservation=5.81MB spill-buffer=64.00KB thread-reservation=0 cost=22
| |
| 11:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=3 row-size=8B cardinality=7.30K cost=9700
-| | in pipelines: 03(GETNEXT)
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=1 row-size=32B cardinality=22 cost=20
+| | in pipelines: 01(GETNEXT)
| |
| F03:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
-| 03:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| HDFS partitions=1/1 files=1 size=2.17MB
-| predicates: d_date <= DATE '2000-06-21', d_date >= DATE '2000-04-22'
+| Per-Instance Resources: mem-estimate=16.14MB mem-reservation=16.00KB thread-reservation=1
+| max-parallelism=1 segment-costs=[8]
+| 01:SCAN HDFS [tpcds_partitioned_parquet_snap.warehouse, RANDOM]
+| HDFS partitions=1/1 files=1 size=4.38KB
| stored statistics:
-| table: rows=73.05K size=2.17MB
+| table: rows=22 size=5.99KB
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=73.05K
-| parquet statistics predicates: d_date <= DATE '2000-06-21', d_date >= DATE '2000-04-22'
-| parquet dictionary predicates: d_date <= DATE '2000-06-21', d_date >= DATE '2000-04-22'
-| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=3 row-size=8B cardinality=7.30K cost=12520
-| in pipelines: 03(GETNEXT)
+| extrapolated-rows=disabled max-scan-range-rows=22
+| mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=0
+| tuple-ids=1 row-size=32B cardinality=22 cost=6
+| in pipelines: 01(GETNEXT)
|
05:HASH JOIN [INNER JOIN, BROADCAST]
| hash-table-id=01
-| hash predicates: inv_warehouse_sk = w_warehouse_sk
-| fk/pk conjuncts: inv_warehouse_sk = w_warehouse_sk
+| hash predicates: inv_date_sk = d_date_sk
+| fk/pk conjuncts: inv_date_sk = d_date_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,2,1 row-size=84B cardinality=103.30M cost=45216313
-| in pipelines: 00(GETNEXT), 01(OPEN)
+| tuple-ids=0,2,3 row-size=60B cardinality=24.14M cost=10567797
+| in pipelines: 00(GETNEXT), 03(OPEN)
|
|--F07:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=20.39MB mem-reservation=20.38MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[42]
+| | Per-Instance Resources: mem-estimate=6.83MB mem-reservation=6.81MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[141]
| JOIN BUILD
| | join-table-id=01 plan-id=02 cohort-id=01
-| | build expressions: w_warehouse_sk
-| | runtime filters: RF002[bloom] <- w_warehouse_sk, RF003[min_max] <- w_warehouse_sk
-| | mem-estimate=19.38MB mem-reservation=19.38MB spill-buffer=64.00KB thread-reservation=0 cost=22
+| | build expressions: d_date_sk
+| | runtime filters: RF002[bloom] <- d_date_sk
+| | mem-estimate=5.81MB mem-reservation=5.81MB spill-buffer=64.00KB thread-reservation=0 cost=61
| |
| 10:EXCHANGE [BROADCAST]
| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=1 row-size=32B cardinality=22 cost=20
-| | in pipelines: 01(GETNEXT)
+| | tuple-ids=3 row-size=8B cardinality=61 cost=80
+| | in pipelines: 03(GETNEXT)
| |
| F02:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=16.14MB mem-reservation=16.00KB thread-reservation=1
-| max-parallelism=1 segment-costs=[8]
-| 01:SCAN HDFS [tpcds_partitioned_parquet_snap.warehouse, RANDOM]
-| HDFS partitions=1/1 files=1 size=4.38KB
+| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| max-parallelism=1 segment-costs=[12522]
+| 03:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| HDFS partitions=1/1 files=1 size=2.17MB
+| predicates: d_date <= DATE '2000-06-21', d_date >= DATE '2000-04-22'
| stored statistics:
-| table: rows=22 size=5.99KB
+| table: rows=73.05K size=2.17MB
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=22
-| mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=0
-| tuple-ids=1 row-size=32B cardinality=22 cost=6
-| in pipelines: 01(GETNEXT)
+| extrapolated-rows=disabled max-scan-range-rows=73.05K
+| parquet statistics predicates: d_date <= DATE '2000-06-21', d_date >= DATE '2000-04-22'
+| parquet dictionary predicates: d_date <= DATE '2000-06-21', d_date >= DATE '2000-04-22'
+| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| tuple-ids=3 row-size=8B cardinality=61 cost=12520
+| in pipelines: 03(GETNEXT)
|
04:HASH JOIN [INNER JOIN, BROADCAST]
| hash-table-id=02
| hash predicates: inv_item_sk = i_item_sk
| fk/pk conjuncts: inv_item_sk = i_item_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
-| tuple-ids=0,2 row-size=52B cardinality=103.30M cost=45216313
+| tuple-ids=0,2 row-size=52B cardinality=24.14M(filtered from 103.30M) cost=24911665
| in pipelines: 00(GETNEXT), 02(OPEN)
|
|--F08:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=31.14MB mem-reservation=29.75MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | Per-Instance Resources: mem-estimate=11.02MB mem-reservation=9.62MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[83840]
| JOIN BUILD
| | join-table-id=02 plan-id=03 cohort-id=01
| | build expressions: i_item_sk
| | runtime filters: RF004[bloom] <- i_item_sk, RF005[min_max] <- i_item_sk
-| | mem-estimate=28.75MB mem-reservation=28.75MB spill-buffer=128.00KB thread-reservation=0 cost=36000
+| | mem-estimate=8.62MB mem-reservation=8.62MB spill-buffer=128.00KB thread-reservation=0 cost=36000
| |
| 09:EXCHANGE [BROADCAST]
| | mem-estimate=1.39MB mem-reservation=0B thread-reservation=0
@@ -287,21 +287,21 @@ max-parallelism=100 segment-costs=[990776711, 20089249] cpu-comparison-result=10
|
00:SCAN HDFS [tpcds_partitioned_parquet_snap.inventory, RANDOM]
HDFS partitions=261/261 files=261 size=44.10MB
- runtime filters: RF003[min_max] -> inv_warehouse_sk, RF005[min_max] -> inv_item_sk, RF000[bloom] -> inv_date_sk, RF002[bloom] -> inv_warehouse_sk, RF004[bloom] -> inv_item_sk
+ runtime filters: RF001[min_max] -> inv_warehouse_sk, RF005[min_max] -> inv_item_sk, RF000[bloom] -> inv_warehouse_sk, RF002[bloom] -> inv_date_sk, RF004[bloom] -> inv_item_sk
stored statistics:
table: rows=1.03G size=5.10GB
partitions: 261/261 rows=1.03G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=3.98M
+ extrapolated-rows=disabled max-scan-range-rows=3.98M est-scan-range=61(filtered from 261)
mem-estimate=16.00MB mem-reservation=256.00KB thread-reservation=0
- tuple-ids=0 row-size=16B cardinality=103.30M(filtered from 1.03G) cost=238132224
+ tuple-ids=0 row-size=16B cardinality=24.14M(filtered from 1.03G) cost=55655424
in pipelines: 00(GETNEXT)
---- PARALLELPLANS
-Max Per-Host Resource Reservation: Memory=455.52MB Threads=18
-Per-Host Resource Estimates: Memory=990MB
+Max Per-Host Resource Reservation: Memory=168.52MB Threads=11
+Per-Host Resource Estimates: Memory=328MB
F05:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=4.07MB mem-reservation=4.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[439] cpu-comparison-result=100 [max(1 (self) vs 100 (sum children))]
+| max-parallelism=1 segment-costs=[439] cpu-comparison-result=36 [max(1 (self) vs 36 (sum children))]
PLAN-ROOT SINK
| output exprs: w_warehouse_name, i_item_id, inv_before, inv_after
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=400
@@ -314,8 +314,8 @@ PLAN-ROOT SINK
| in pipelines: 08(GETNEXT)
|
F04:PLAN FRAGMENT [HASH(w_warehouse_name,i_item_id)] hosts=10 instances=10 (adjusted from 120)
-Per-Instance Resources: mem-estimate=51.43MB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=10 segment-costs=[24999660, 4080496, 220] cpu-comparison-result=100 [max(10 (self) vs 100 (sum children))]
+Per-Instance Resources: mem-estimate=46.23MB mem-reservation=34.00MB thread-reservation=1
+max-parallelism=10 segment-costs=[24999660, 4080496, 220] cpu-comparison-result=36 [max(10 (self) vs 36 (sum children))]
08:TOP-N [LIMIT=100]
| order by: w_warehouse_name ASC, i_item_id ASC
| mem-estimate=7.04KB mem-reservation=0B thread-reservation=0
@@ -331,110 +331,110 @@ max-parallelism=10 segment-costs=[24999660, 4080496, 220] cpu-comparison-result=
| in pipelines: 13(GETNEXT), 00(OPEN)
|
12:EXCHANGE [HASH(w_warehouse_name,i_item_id)]
-| mem-estimate=17.43MB mem-reservation=0B thread-reservation=0
+| mem-estimate=12.23MB mem-reservation=0B thread-reservation=0
| tuple-ids=4 row-size=72B cardinality=3.90M cost=1724492
| in pipelines: 00(GETNEXT)
|
-F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=100 (adjusted from 120)
+F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=30 (adjusted from 120)
Per-Host Shared Resources: mem-estimate=3.00MB mem-reservation=3.00MB thread-reservation=0 runtime-filters-memory=3.00MB
-Per-Instance Resources: mem-estimate=81.07MB mem-reservation=34.25MB thread-reservation=1
-max-parallelism=100 segment-costs=[990776711, 20089249] cpu-comparison-result=100 [max(100 (self) vs 36 (sum children))]
+Per-Instance Resources: mem-estimate=67.35MB mem-reservation=34.25MB thread-reservation=1
+max-parallelism=30 segment-costs=[245904713, 20089249] cpu-comparison-result=36 [max(30 (self) vs 36 (sum children))]
07:AGGREGATE [STREAMING]
| output: sum(CAST(CASE WHEN d_date < DATE '2000-05-22' THEN inv_quantity_on_hand ELSE CAST(0 AS INT) END AS BIGINT)), sum(CAST(CASE WHEN d_date >= DATE '2000-05-22' THEN inv_quantity_on_hand ELSE CAST(0 AS INT) END AS BIGINT))
| group by: w_warehouse_name, i_item_id
-| mem-estimate=62.10MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=4 row-size=72B cardinality=3.90M cost=616995548
+| mem-estimate=48.38MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=4 row-size=72B cardinality=3.90M cost=144202030
| in pipelines: 00(GETNEXT)
|
06:HASH JOIN [INNER JOIN, BROADCAST]
| hash-table-id=00
-| hash predicates: inv_date_sk = d_date_sk
-| fk/pk conjuncts: inv_date_sk = d_date_sk
+| hash predicates: inv_warehouse_sk = w_warehouse_sk
+| fk/pk conjuncts: inv_warehouse_sk = w_warehouse_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,2,1,3 row-size=92B cardinality=103.30M cost=45216313
-| in pipelines: 00(GETNEXT), 03(OPEN)
+| tuple-ids=0,2,3,1 row-size=92B cardinality=24.14M cost=10567797
+| in pipelines: 00(GETNEXT), 01(OPEN)
|
|--F06:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=20.44MB mem-reservation=20.38MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
+| | Per-Instance Resources: mem-estimate=6.83MB mem-reservation=6.81MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[42]
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
-| | build expressions: d_date_sk
-| | runtime filters: RF000[bloom] <- d_date_sk
-| | mem-estimate=19.38MB mem-reservation=19.38MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | build expressions: w_warehouse_sk
+| | runtime filters: RF000[bloom] <- w_warehouse_sk, RF001[min_max] <- w_warehouse_sk
+| | mem-estimate=5.81MB mem-reservation=5.81MB spill-buffer=64.00KB thread-reservation=0 cost=22
| |
| 11:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=3 row-size=8B cardinality=7.30K cost=9700
-| | in pipelines: 03(GETNEXT)
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=1 row-size=32B cardinality=22 cost=20
+| | in pipelines: 01(GETNEXT)
| |
| F03:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
-| 03:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| HDFS partitions=1/1 files=1 size=2.17MB
-| predicates: d_date <= DATE '2000-06-21', d_date >= DATE '2000-04-22'
+| Per-Instance Resources: mem-estimate=16.14MB mem-reservation=16.00KB thread-reservation=1
+| max-parallelism=1 segment-costs=[8]
+| 01:SCAN HDFS [tpcds_partitioned_parquet_snap.warehouse, RANDOM]
+| HDFS partitions=1/1 files=1 size=4.38KB
| stored statistics:
-| table: rows=73.05K size=2.17MB
+| table: rows=22 size=5.99KB
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=73.05K
-| parquet statistics predicates: d_date <= DATE '2000-06-21', d_date >= DATE '2000-04-22'
-| parquet dictionary predicates: d_date <= DATE '2000-06-21', d_date >= DATE '2000-04-22'
-| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=3 row-size=8B cardinality=7.30K cost=12520
-| in pipelines: 03(GETNEXT)
+| extrapolated-rows=disabled max-scan-range-rows=22
+| mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=0
+| tuple-ids=1 row-size=32B cardinality=22 cost=6
+| in pipelines: 01(GETNEXT)
|
05:HASH JOIN [INNER JOIN, BROADCAST]
| hash-table-id=01
-| hash predicates: inv_warehouse_sk = w_warehouse_sk
-| fk/pk conjuncts: inv_warehouse_sk = w_warehouse_sk
+| hash predicates: inv_date_sk = d_date_sk
+| fk/pk conjuncts: inv_date_sk = d_date_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,2,1 row-size=84B cardinality=103.30M cost=45216313
-| in pipelines: 00(GETNEXT), 01(OPEN)
+| tuple-ids=0,2,3 row-size=60B cardinality=24.14M cost=10567797
+| in pipelines: 00(GETNEXT), 03(OPEN)
|
|--F07:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=20.39MB mem-reservation=20.38MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[42]
+| | Per-Instance Resources: mem-estimate=6.83MB mem-reservation=6.81MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[141]
| JOIN BUILD
| | join-table-id=01 plan-id=02 cohort-id=01
-| | build expressions: w_warehouse_sk
-| | runtime filters: RF002[bloom] <- w_warehouse_sk, RF003[min_max] <- w_warehouse_sk
-| | mem-estimate=19.38MB mem-reservation=19.38MB spill-buffer=64.00KB thread-reservation=0 cost=22
+| | build expressions: d_date_sk
+| | runtime filters: RF002[bloom] <- d_date_sk
+| | mem-estimate=5.81MB mem-reservation=5.81MB spill-buffer=64.00KB thread-reservation=0 cost=61
| |
| 10:EXCHANGE [BROADCAST]
| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=1 row-size=32B cardinality=22 cost=20
-| | in pipelines: 01(GETNEXT)
+| | tuple-ids=3 row-size=8B cardinality=61 cost=80
+| | in pipelines: 03(GETNEXT)
| |
| F02:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=16.14MB mem-reservation=16.00KB thread-reservation=1
-| max-parallelism=1 segment-costs=[8]
-| 01:SCAN HDFS [tpcds_partitioned_parquet_snap.warehouse, RANDOM]
-| HDFS partitions=1/1 files=1 size=4.38KB
+| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| max-parallelism=1 segment-costs=[12522]
+| 03:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| HDFS partitions=1/1 files=1 size=2.17MB
+| predicates: d_date <= DATE '2000-06-21', d_date >= DATE '2000-04-22'
| stored statistics:
-| table: rows=22 size=5.99KB
+| table: rows=73.05K size=2.17MB
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=22
-| mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=0
-| tuple-ids=1 row-size=32B cardinality=22 cost=6
-| in pipelines: 01(GETNEXT)
+| extrapolated-rows=disabled max-scan-range-rows=73.05K
+| parquet statistics predicates: d_date <= DATE '2000-06-21', d_date >= DATE '2000-04-22'
+| parquet dictionary predicates: d_date <= DATE '2000-06-21', d_date >= DATE '2000-04-22'
+| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| tuple-ids=3 row-size=8B cardinality=61 cost=12520
+| in pipelines: 03(GETNEXT)
|
04:HASH JOIN [INNER JOIN, BROADCAST]
| hash-table-id=02
| hash predicates: inv_item_sk = i_item_sk
| fk/pk conjuncts: inv_item_sk = i_item_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
-| tuple-ids=0,2 row-size=52B cardinality=103.30M cost=45216313
+| tuple-ids=0,2 row-size=52B cardinality=24.14M(filtered from 103.30M) cost=24911665
| in pipelines: 00(GETNEXT), 02(OPEN)
|
|--F08:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=31.14MB mem-reservation=29.75MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | Per-Instance Resources: mem-estimate=11.02MB mem-reservation=9.62MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[83840]
| JOIN BUILD
| | join-table-id=02 plan-id=03 cohort-id=01
| | build expressions: i_item_sk
| | runtime filters: RF004[bloom] <- i_item_sk, RF005[min_max] <- i_item_sk
-| | mem-estimate=28.75MB mem-reservation=28.75MB spill-buffer=128.00KB thread-reservation=0 cost=36000
+| | mem-estimate=8.62MB mem-reservation=8.62MB spill-buffer=128.00KB thread-reservation=0 cost=36000
| |
| 09:EXCHANGE [BROADCAST]
| | mem-estimate=1.39MB mem-reservation=0B thread-reservation=0
@@ -459,13 +459,13 @@ max-parallelism=100 segment-costs=[990776711, 20089249] cpu-comparison-result=10
|
00:SCAN HDFS [tpcds_partitioned_parquet_snap.inventory, RANDOM]
HDFS partitions=261/261 files=261 size=44.10MB
- runtime filters: RF003[min_max] -> inv_warehouse_sk, RF005[min_max] -> inv_item_sk, RF000[bloom] -> inv_date_sk, RF002[bloom] -> inv_warehouse_sk, RF004[bloom] -> inv_item_sk
+ runtime filters: RF001[min_max] -> inv_warehouse_sk, RF005[min_max] -> inv_item_sk, RF000[bloom] -> inv_warehouse_sk, RF002[bloom] -> inv_date_sk, RF004[bloom] -> inv_item_sk
stored statistics:
table: rows=1.03G size=5.10GB
partitions: 261/261 rows=1.03G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=3.98M
+ extrapolated-rows=disabled max-scan-range-rows=3.98M est-scan-range=61(filtered from 261)
mem-estimate=16.00MB mem-reservation=256.00KB thread-reservation=0
- tuple-ids=0 row-size=16B cardinality=103.30M(filtered from 1.03G) cost=238132224
+ tuple-ids=0 row-size=16B cardinality=24.14M(filtered from 1.03G) cost=55655424
in pipelines: 00(GETNEXT)
====
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q32.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q32.test
index 755a3c153..b7db278c3 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q32.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q32.test
@@ -32,7 +32,7 @@ Max Per-Host Resource Reservation: Memory=61.38MB Threads=1
Per-Host Resource Estimates: Memory=107MB
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=106.88MB mem-reservation=61.38MB thread-reservation=1 runtime-filters-memory=4.00MB
-| max-parallelism=1 segment-costs=[3901989300, 753162393, 0]
+| max-parallelism=1 segment-costs=[194831391, 38609667, 0]
PLAN-ROOT SINK
| output exprs: sum(cs_ext_discount_amt)
| mem-estimate=0B mem-reservation=0B thread-reservation=0 cost=0
@@ -41,7 +41,7 @@ PLAN-ROOT SINK
| output: sum(cs_ext_discount_amt)
| limit: 100
| mem-estimate=16.00KB mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=8 row-size=16B cardinality=1 cost=314848
+| tuple-ids=8 row-size=16B cardinality=1 cost=15647
| in pipelines: 10(GETNEXT), 00(OPEN)
|
09:HASH JOIN [LEFT SEMI JOIN]
@@ -49,14 +49,14 @@ PLAN-ROOT SINK
| other join predicates: cs_ext_discount_amt > CAST(1.3 AS DECIMAL(2,1)) * avg(cs_ext_discount_amt)
| runtime filters: RF000[bloom] <- cs_item_sk, RF001[min_max] <- cs_item_sk
| mem-estimate=34.00MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,1,2 row-size=28B cardinality=4.50M cost=2328881
+| tuple-ids=0,1,2 row-size=28B cardinality=223.54K cost=458023
| in pipelines: 00(GETNEXT), 06(OPEN)
|
|--06:AGGREGATE [FINALIZE]
| | output: avg(cs_ext_discount_amt)
| | group by: cs_item_sk
| | mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB thread-reservation=0
-| | tuple-ids=6 row-size=12B cardinality=360.18K cost=1264561870
+| | tuple-ids=6 row-size=12B cardinality=360.18K cost=63740631
| | in pipelines: 06(GETNEXT), 03(OPEN)
| |
| 05:HASH JOIN [INNER JOIN]
@@ -64,7 +64,7 @@ PLAN-ROOT SINK
| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
| | runtime filters: RF006[bloom] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=3,4 row-size=20B cardinality=4.32G cost=1890905436
+| | tuple-ids=3,4 row-size=20B cardinality=214.71M cost=93977005
| | in pipelines: 03(GETNEXT), 04(OPEN)
| |
| |--04:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
@@ -77,7 +77,7 @@ PLAN-ROOT SINK
| | parquet statistics predicates: d_date <= DATE '1998-05-04', d_date >= DATE '1998-02-03'
| | parquet dictionary predicates: d_date <= DATE '1998-05-04', d_date >= DATE '1998-02-03'
| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=4 row-size=8B cardinality=7.30K cost=12520
+| | tuple-ids=4 row-size=8B cardinality=91 cost=12520
| | in pipelines: 04(GETNEXT)
| |
| 03:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales]
@@ -87,9 +87,9 @@ PLAN-ROOT SINK
| table: rows=4.32G size=280.96GB
| partitions: 1831/1831 rows=4.32G
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=21.52M
+| extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=91(filtered from 1831)
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=3 row-size=12B cardinality=4.32G cost=746509474
+| tuple-ids=3 row-size=12B cardinality=214.71M(filtered from 4.32G) cost=37101235
| in pipelines: 03(GETNEXT)
|
08:HASH JOIN [INNER JOIN]
@@ -97,7 +97,7 @@ PLAN-ROOT SINK
| fk/pk conjuncts: cs_sold_date_sk = d_date_sk
| runtime filters: RF002[bloom] <- d_date_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1,2 row-size=28B cardinality=4.50M cost=1976006
+| tuple-ids=0,1,2 row-size=28B cardinality=223.54K cost=97934
| in pipelines: 00(GETNEXT), 02(OPEN)
|
|--02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
@@ -110,7 +110,7 @@ PLAN-ROOT SINK
| parquet statistics predicates: d_date <= DATE '1998-05-04', d_date >= DATE '1998-02-03'
| parquet dictionary predicates: d_date <= DATE '1998-05-04', d_date >= DATE '1998-02-03'
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
+| tuple-ids=2 row-size=8B cardinality=91 cost=12520
| in pipelines: 02(GETNEXT)
|
07:HASH JOIN [INNER JOIN]
@@ -118,7 +118,7 @@ PLAN-ROOT SINK
| fk/pk conjuncts: cs_item_sk = i_item_sk
| runtime filters: RF004[bloom] <- i_item_sk, RF005[min_max] <- i_item_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1 row-size=20B cardinality=4.50M cost=1969076
+| tuple-ids=0,1 row-size=20B cardinality=223.54K(filtered from 4.50M) cost=872720
| in pipelines: 00(GETNEXT), 01(OPEN)
|
|--01:SCAN HDFS [tpcds_partitioned_parquet_snap.item]
@@ -142,16 +142,16 @@ PLAN-ROOT SINK
table: rows=4.32G size=280.96GB
partitions: 1831/1831 rows=4.32G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=21.52M
+ extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=91(filtered from 1831)
mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
- tuple-ids=0 row-size=12B cardinality=4.50M(filtered from 4.32G) cost=746509474
+ tuple-ids=0 row-size=12B cardinality=223.54K(filtered from 4.32G) cost=37101235
in pipelines: 00(GETNEXT)
---- DISTRIBUTEDPLAN
-Max Per-Host Resource Reservation: Memory=563.69MB Threads=29
-Per-Host Resource Estimates: Memory=932MB
+Max Per-Host Resource Reservation: Memory=100.94MB Threads=14
+Per-Host Resource Estimates: Memory=235MB
F06:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=32.00KB mem-reservation=0B thread-reservation=1
-| max-parallelism=1 segment-costs=[0, 0] cpu-comparison-result=145 [max(1 (self) vs 145 (sum children))]
+| max-parallelism=1 segment-costs=[0, 0] cpu-comparison-result=65 [max(1 (self) vs 65 (sum children))]
PLAN-ROOT SINK
| output exprs: sum(cs_ext_discount_amt)
| mem-estimate=0B mem-reservation=0B thread-reservation=0 cost=0
@@ -168,41 +168,32 @@ PLAN-ROOT SINK
| tuple-ids=8 row-size=16B cardinality=1 cost=0
| in pipelines: 10(GETNEXT)
|
-F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=80 (adjusted from 120)
-Per-Host Shared Resources: mem-estimate=3.00MB mem-reservation=3.00MB thread-reservation=0 runtime-filters-memory=3.00MB
-Per-Instance Resources: mem-estimate=16.02MB mem-reservation=1.00MB thread-reservation=1
-max-parallelism=80 segment-costs=[752730425, 1] cpu-comparison-result=145 [max(80 (self) vs 145 (sum children))]
+F05:PLAN FRAGMENT [HASH(cs_item_sk)] hosts=10 instances=10 (adjusted from 120)
+Per-Instance Resources: mem-estimate=1.00MB mem-reservation=0B thread-reservation=1
+max-parallelism=10 segment-costs=[183570, 1] cpu-comparison-result=65 [max(20 (self) vs 65 (sum children))]
10:AGGREGATE
| output: sum(cs_ext_discount_amt)
| mem-estimate=16.00KB mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=8 row-size=16B cardinality=1 cost=314848
+| tuple-ids=8 row-size=16B cardinality=1 cost=15647
| in pipelines: 10(GETNEXT), 00(OPEN)
|
-09:HASH JOIN [LEFT SEMI JOIN, BROADCAST]
+09:HASH JOIN [LEFT SEMI JOIN, PARTITIONED]
| hash-table-id=00
| hash predicates: i_item_sk = cs_item_sk
| other join predicates: cs_ext_discount_amt > CAST(1.3 AS DECIMAL(2,1)) * avg(cs_ext_discount_amt)
-| mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,1,2 row-size=28B cardinality=4.50M cost=1968701
+| mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
+| tuple-ids=0,1,2 row-size=28B cardinality=223.54K cost=97843
| in pipelines: 00(GETNEXT), 15(OPEN)
|
-|--F07:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=277.28MB mem-reservation=273.00MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[838850] cpu-comparison-result=120 [max(10 (self) vs 120 (sum children))]
+|--F07:PLAN FRAGMENT [HASH(cs_item_sk)] hosts=10 instances=10 (adjusted from 120)
+| | Per-Instance Resources: mem-estimate=13.88MB mem-reservation=5.81MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[1100400, 360180] cpu-comparison-result=40 [max(10 (self) vs 40 (sum children))]
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: cs_item_sk
| | runtime filters: RF000[bloom] <- cs_item_sk, RF001[min_max] <- cs_item_sk
-| | mem-estimate=272.00MB mem-reservation=272.00MB spill-buffer=2.00MB thread-reservation=0 cost=360180
-| |
-| 16:EXCHANGE [BROADCAST]
-| | mem-estimate=4.28MB mem-reservation=0B thread-reservation=0
-| | tuple-ids=6 row-size=12B cardinality=360.18K cost=478670
-| | in pipelines: 15(GETNEXT)
+| | mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0 cost=360180
| |
-| F05:PLAN FRAGMENT [HASH(cs_item_sk)] hosts=10 instances=10 (adjusted from 120)
-| Per-Instance Resources: mem-estimate=12.29MB mem-reservation=1.94MB thread-reservation=1
-| max-parallelism=10 segment-costs=[1100400, 19449] cpu-comparison-result=120 [max(10 (self) vs 120 (sum children))]
| 15:AGGREGATE [FINALIZE]
| | output: avg:merge(cs_ext_discount_amt)
| | group by: cs_item_sk
@@ -211,19 +202,19 @@ max-parallelism=80 segment-costs=[752730425, 1] cpu-comparison-result=145 [max(8
| | in pipelines: 15(GETNEXT), 03(OPEN)
| |
| 14:EXCHANGE [HASH(cs_item_sk)]
-| | mem-estimate=2.29MB mem-reservation=0B thread-reservation=0
+| | mem-estimate=1.04MB mem-reservation=0B thread-reservation=0
| | tuple-ids=5 row-size=12B cardinality=360.18K cost=59897
| | in pipelines: 03(GETNEXT)
| |
-| F03:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+| F03:PLAN FRAGMENT [RANDOM] hosts=10 instances=40 (adjusted from 120)
| Per-Host Shared Resources: mem-estimate=1.00MB mem-reservation=1.00MB thread-reservation=0 runtime-filters-memory=1.00MB
| Per-Instance Resources: mem-estimate=33.62MB mem-reservation=18.00MB thread-reservation=1
-| max-parallelism=410 segment-costs=[4013717769, 463911] cpu-comparison-result=120 [max(120 (self) vs 11 (sum children))]
+| max-parallelism=40 segment-costs=[305782724, 463911] cpu-comparison-result=40 [max(40 (self) vs 11 (sum children))]
| 06:AGGREGATE [STREAMING]
| | output: avg(cs_ext_discount_amt)
| | group by: cs_item_sk
| | mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB thread-reservation=0
-| | tuple-ids=5 row-size=12B cardinality=360.18K cost=1376310164
+| | tuple-ids=5 row-size=12B cardinality=360.18K cost=174704575
| | in pipelines: 03(GETNEXT)
| |
| 05:HASH JOIN [INNER JOIN, BROADCAST]
@@ -231,26 +222,26 @@ max-parallelism=80 segment-costs=[752730425, 1] cpu-comparison-result=145 [max(8
| | hash predicates: cs_sold_date_sk = d_date_sk
| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=3,4 row-size=20B cardinality=4.32G cost=1890898131
+| | tuple-ids=3,4 row-size=20B cardinality=214.71M cost=93976914
| | in pipelines: 03(GETNEXT), 04(OPEN)
| |
| |--F08:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[17005]
+| | | Per-Instance Resources: mem-estimate=8.77MB mem-reservation=8.75MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[211]
| | JOIN BUILD
| | | join-table-id=01 plan-id=02 cohort-id=02
| | | build expressions: d_date_sk
| | | runtime filters: RF006[bloom] <- d_date_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | | mem-estimate=7.75MB mem-reservation=7.75MB spill-buffer=64.00KB thread-reservation=0 cost=91
| | |
| | 13:EXCHANGE [BROADCAST]
-| | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=4 row-size=8B cardinality=7.30K cost=9700
+| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=4 row-size=8B cardinality=91 cost=120
| | | in pipelines: 04(GETNEXT)
| | |
| | F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[12835]
+| | max-parallelism=1 segment-costs=[12523]
| | 04:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| | HDFS partitions=1/1 files=1 size=2.17MB
| | predicates: d_date <= DATE '1998-05-04', d_date >= DATE '1998-02-03'
@@ -261,7 +252,7 @@ max-parallelism=80 segment-costs=[752730425, 1] cpu-comparison-result=145 [max(8
| | parquet statistics predicates: d_date <= DATE '1998-05-04', d_date >= DATE '1998-02-03'
| | parquet dictionary predicates: d_date <= DATE '1998-05-04', d_date >= DATE '1998-02-03'
| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=4 row-size=8B cardinality=7.30K cost=12520
+| | tuple-ids=4 row-size=8B cardinality=91 cost=12520
| | in pipelines: 04(GETNEXT)
| |
| 03:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales, RANDOM]
@@ -271,36 +262,45 @@ max-parallelism=80 segment-costs=[752730425, 1] cpu-comparison-result=145 [max(8
| table: rows=4.32G size=280.96GB
| partitions: 1831/1831 rows=4.32G
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=21.52M
+| extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=91(filtered from 1831)
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=3 row-size=12B cardinality=4.32G cost=746509474
+| tuple-ids=3 row-size=12B cardinality=214.71M(filtered from 4.32G) cost=37101235
| in pipelines: 03(GETNEXT)
|
+16:EXCHANGE [HASH(i_item_sk)]
+| mem-estimate=1011.24KB mem-reservation=0B thread-reservation=0
+| tuple-ids=0,1,2 row-size=28B cardinality=223.54K cost=70080
+| in pipelines: 00(GETNEXT)
+|
+F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=10 (adjusted from 120)
+Per-Host Shared Resources: mem-estimate=3.00MB mem-reservation=3.00MB thread-reservation=0 runtime-filters-memory=3.00MB
+Per-Instance Resources: mem-estimate=17.56MB mem-reservation=1.00MB thread-reservation=1
+max-parallelism=10 segment-costs=[38820017]
08:HASH JOIN [INNER JOIN, BROADCAST]
| hash-table-id=02
| hash predicates: cs_sold_date_sk = d_date_sk
| fk/pk conjuncts: cs_sold_date_sk = d_date_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1,2 row-size=28B cardinality=4.50M cost=1968701
+| tuple-ids=0,1,2 row-size=28B cardinality=223.54K cost=97843
| in pipelines: 00(GETNEXT), 02(OPEN)
|
|--F09:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=16.57MB mem-reservation=16.50MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
+| | Per-Instance Resources: mem-estimate=2.95MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[211]
| JOIN BUILD
| | join-table-id=02 plan-id=03 cohort-id=01
| | build expressions: d_date_sk
| | runtime filters: RF002[bloom] <- d_date_sk
-| | mem-estimate=15.50MB mem-reservation=15.50MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=91
| |
| 12:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=2 row-size=8B cardinality=7.30K cost=9700
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=2 row-size=8B cardinality=91 cost=120
| | in pipelines: 02(GETNEXT)
| |
| F02:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
+| max-parallelism=1 segment-costs=[12523]
| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| HDFS partitions=1/1 files=1 size=2.17MB
| predicates: d_date <= DATE '1998-05-04', d_date >= DATE '1998-02-03'
@@ -311,7 +311,7 @@ max-parallelism=80 segment-costs=[752730425, 1] cpu-comparison-result=145 [max(8
| parquet statistics predicates: d_date <= DATE '1998-05-04', d_date >= DATE '1998-02-03'
| parquet dictionary predicates: d_date <= DATE '1998-05-04', d_date >= DATE '1998-02-03'
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
+| tuple-ids=2 row-size=8B cardinality=91 cost=12520
| in pipelines: 02(GETNEXT)
|
07:HASH JOIN [INNER JOIN, BROADCAST]
@@ -319,17 +319,17 @@ max-parallelism=80 segment-costs=[752730425, 1] cpu-comparison-result=145 [max(8
| hash predicates: cs_item_sk = i_item_sk
| fk/pk conjuncts: cs_item_sk = i_item_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1 row-size=20B cardinality=4.50M cost=1968701
+| tuple-ids=0,1 row-size=20B cardinality=223.54K(filtered from 4.50M) cost=872345
| in pipelines: 00(GETNEXT), 01(OPEN)
|
|--F10:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=16.52MB mem-reservation=16.50MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | Per-Instance Resources: mem-estimate=2.96MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[865]
| JOIN BUILD
| | join-table-id=03 plan-id=04 cohort-id=01
| | build expressions: i_item_sk
| | runtime filters: RF004[bloom] <- i_item_sk, RF005[min_max] <- i_item_sk
-| | mem-estimate=15.50MB mem-reservation=15.50MB spill-buffer=64.00KB thread-reservation=0 cost=375
+| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=375
| |
| 11:EXCHANGE [BROADCAST]
| | mem-estimate=20.51KB mem-reservation=0B thread-reservation=0
@@ -361,16 +361,16 @@ max-parallelism=80 segment-costs=[752730425, 1] cpu-comparison-result=145 [max(8
table: rows=4.32G size=280.96GB
partitions: 1831/1831 rows=4.32G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=21.52M
+ extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=91(filtered from 1831)
mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
- tuple-ids=0 row-size=12B cardinality=4.50M(filtered from 4.32G) cost=746509474
+ tuple-ids=0 row-size=12B cardinality=223.54K(filtered from 4.32G) cost=37101235
in pipelines: 00(GETNEXT)
---- PARALLELPLANS
-Max Per-Host Resource Reservation: Memory=294.56MB Threads=29
-Per-Host Resource Estimates: Memory=682MB
+Max Per-Host Resource Reservation: Memory=100.94MB Threads=14
+Per-Host Resource Estimates: Memory=235MB
F06:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=32.00KB mem-reservation=0B thread-reservation=1
-| max-parallelism=1 segment-costs=[0, 0] cpu-comparison-result=145 [max(1 (self) vs 145 (sum children))]
+| max-parallelism=1 segment-costs=[0, 0] cpu-comparison-result=65 [max(1 (self) vs 65 (sum children))]
PLAN-ROOT SINK
| output exprs: sum(cs_ext_discount_amt)
| mem-estimate=0B mem-reservation=0B thread-reservation=0 cost=0
@@ -388,12 +388,12 @@ PLAN-ROOT SINK
| in pipelines: 10(GETNEXT)
|
F05:PLAN FRAGMENT [HASH(cs_item_sk)] hosts=10 instances=10 (adjusted from 120)
-Per-Instance Resources: mem-estimate=13.14MB mem-reservation=0B thread-reservation=1
-max-parallelism=10 segment-costs=[3693619, 1] cpu-comparison-result=145 [max(90 (self) vs 145 (sum children))]
+Per-Instance Resources: mem-estimate=1.00MB mem-reservation=0B thread-reservation=1
+max-parallelism=10 segment-costs=[183570, 1] cpu-comparison-result=65 [max(20 (self) vs 65 (sum children))]
10:AGGREGATE
| output: sum(cs_ext_discount_amt)
| mem-estimate=16.00KB mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=8 row-size=16B cardinality=1 cost=314848
+| tuple-ids=8 row-size=16B cardinality=1 cost=15647
| in pipelines: 10(GETNEXT), 00(OPEN)
|
09:HASH JOIN [LEFT SEMI JOIN, PARTITIONED]
@@ -401,12 +401,12 @@ max-parallelism=10 segment-costs=[3693619, 1] cpu-comparison-result=145 [max(90
| hash predicates: i_item_sk = cs_item_sk
| other join predicates: cs_ext_discount_amt > CAST(1.3 AS DECIMAL(2,1)) * avg(cs_ext_discount_amt)
| mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
-| tuple-ids=0,1,2 row-size=28B cardinality=4.50M cost=1968701
+| tuple-ids=0,1,2 row-size=28B cardinality=223.54K cost=97843
| in pipelines: 00(GETNEXT), 15(OPEN)
|
|--F07:PLAN FRAGMENT [HASH(cs_item_sk)] hosts=10 instances=10 (adjusted from 120)
| | Per-Instance Resources: mem-estimate=13.88MB mem-reservation=5.81MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[1100400, 360180] cpu-comparison-result=120 [max(10 (self) vs 120 (sum children))]
+| | max-parallelism=10 segment-costs=[1100400, 360180] cpu-comparison-result=40 [max(10 (self) vs 40 (sum children))]
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: cs_item_sk
@@ -421,19 +421,19 @@ max-parallelism=10 segment-costs=[3693619, 1] cpu-comparison-result=145 [max(90
| | in pipelines: 15(GETNEXT), 03(OPEN)
| |
| 14:EXCHANGE [HASH(cs_item_sk)]
-| | mem-estimate=2.29MB mem-reservation=0B thread-reservation=0
+| | mem-estimate=1.04MB mem-reservation=0B thread-reservation=0
| | tuple-ids=5 row-size=12B cardinality=360.18K cost=59897
| | in pipelines: 03(GETNEXT)
| |
-| F03:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+| F03:PLAN FRAGMENT [RANDOM] hosts=10 instances=40 (adjusted from 120)
| Per-Host Shared Resources: mem-estimate=1.00MB mem-reservation=1.00MB thread-reservation=0 runtime-filters-memory=1.00MB
| Per-Instance Resources: mem-estimate=33.62MB mem-reservation=18.00MB thread-reservation=1
-| max-parallelism=410 segment-costs=[4013717769, 463911] cpu-comparison-result=120 [max(120 (self) vs 11 (sum children))]
+| max-parallelism=40 segment-costs=[305782724, 463911] cpu-comparison-result=40 [max(40 (self) vs 11 (sum children))]
| 06:AGGREGATE [STREAMING]
| | output: avg(cs_ext_discount_amt)
| | group by: cs_item_sk
| | mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB thread-reservation=0
-| | tuple-ids=5 row-size=12B cardinality=360.18K cost=1376310164
+| | tuple-ids=5 row-size=12B cardinality=360.18K cost=174704575
| | in pipelines: 03(GETNEXT)
| |
| 05:HASH JOIN [INNER JOIN, BROADCAST]
@@ -441,26 +441,26 @@ max-parallelism=10 segment-costs=[3693619, 1] cpu-comparison-result=145 [max(90
| | hash predicates: cs_sold_date_sk = d_date_sk
| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=3,4 row-size=20B cardinality=4.32G cost=1890898131
+| | tuple-ids=3,4 row-size=20B cardinality=214.71M cost=93976914
| | in pipelines: 03(GETNEXT), 04(OPEN)
| |
| |--F08:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[17005]
+| | | Per-Instance Resources: mem-estimate=8.77MB mem-reservation=8.75MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[211]
| | JOIN BUILD
| | | join-table-id=01 plan-id=02 cohort-id=02
| | | build expressions: d_date_sk
| | | runtime filters: RF006[bloom] <- d_date_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | | mem-estimate=7.75MB mem-reservation=7.75MB spill-buffer=64.00KB thread-reservation=0 cost=91
| | |
| | 13:EXCHANGE [BROADCAST]
-| | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=4 row-size=8B cardinality=7.30K cost=9700
+| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=4 row-size=8B cardinality=91 cost=120
| | | in pipelines: 04(GETNEXT)
| | |
| | F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[12835]
+| | max-parallelism=1 segment-costs=[12523]
| | 04:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| | HDFS partitions=1/1 files=1 size=2.17MB
| | predicates: d_date <= DATE '1998-05-04', d_date >= DATE '1998-02-03'
@@ -471,7 +471,7 @@ max-parallelism=10 segment-costs=[3693619, 1] cpu-comparison-result=145 [max(90
| | parquet statistics predicates: d_date <= DATE '1998-05-04', d_date >= DATE '1998-02-03'
| | parquet dictionary predicates: d_date <= DATE '1998-05-04', d_date >= DATE '1998-02-03'
| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=4 row-size=8B cardinality=7.30K cost=12520
+| | tuple-ids=4 row-size=8B cardinality=91 cost=12520
| | in pipelines: 04(GETNEXT)
| |
| 03:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales, RANDOM]
@@ -481,45 +481,45 @@ max-parallelism=10 segment-costs=[3693619, 1] cpu-comparison-result=145 [max(90
| table: rows=4.32G size=280.96GB
| partitions: 1831/1831 rows=4.32G
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=21.52M
+| extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=91(filtered from 1831)
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=3 row-size=12B cardinality=4.32G cost=746509474
+| tuple-ids=3 row-size=12B cardinality=214.71M(filtered from 4.32G) cost=37101235
| in pipelines: 03(GETNEXT)
|
16:EXCHANGE [HASH(i_item_sk)]
-| mem-estimate=13.12MB mem-reservation=0B thread-reservation=0
-| tuple-ids=0,1,2 row-size=28B cardinality=4.50M cost=1410070
+| mem-estimate=1011.24KB mem-reservation=0B thread-reservation=0
+| tuple-ids=0,1,2 row-size=28B cardinality=223.54K cost=70080
| in pipelines: 00(GETNEXT)
|
-F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=80 (adjusted from 120)
+F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=10 (adjusted from 120)
Per-Host Shared Resources: mem-estimate=3.00MB mem-reservation=3.00MB thread-reservation=0 runtime-filters-memory=3.00MB
Per-Instance Resources: mem-estimate=17.56MB mem-reservation=1.00MB thread-reservation=1
-max-parallelism=80 segment-costs=[765509219]
+max-parallelism=10 segment-costs=[38820017]
08:HASH JOIN [INNER JOIN, BROADCAST]
| hash-table-id=02
| hash predicates: cs_sold_date_sk = d_date_sk
| fk/pk conjuncts: cs_sold_date_sk = d_date_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1,2 row-size=28B cardinality=4.50M cost=1968701
+| tuple-ids=0,1,2 row-size=28B cardinality=223.54K cost=97843
| in pipelines: 00(GETNEXT), 02(OPEN)
|
|--F09:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=16.57MB mem-reservation=16.50MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
+| | Per-Instance Resources: mem-estimate=2.95MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[211]
| JOIN BUILD
| | join-table-id=02 plan-id=03 cohort-id=01
| | build expressions: d_date_sk
| | runtime filters: RF002[bloom] <- d_date_sk
-| | mem-estimate=15.50MB mem-reservation=15.50MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=91
| |
| 12:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=2 row-size=8B cardinality=7.30K cost=9700
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=2 row-size=8B cardinality=91 cost=120
| | in pipelines: 02(GETNEXT)
| |
| F02:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
+| max-parallelism=1 segment-costs=[12523]
| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| HDFS partitions=1/1 files=1 size=2.17MB
| predicates: d_date <= DATE '1998-05-04', d_date >= DATE '1998-02-03'
@@ -530,7 +530,7 @@ max-parallelism=80 segment-costs=[765509219]
| parquet statistics predicates: d_date <= DATE '1998-05-04', d_date >= DATE '1998-02-03'
| parquet dictionary predicates: d_date <= DATE '1998-05-04', d_date >= DATE '1998-02-03'
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
+| tuple-ids=2 row-size=8B cardinality=91 cost=12520
| in pipelines: 02(GETNEXT)
|
07:HASH JOIN [INNER JOIN, BROADCAST]
@@ -538,17 +538,17 @@ max-parallelism=80 segment-costs=[765509219]
| hash predicates: cs_item_sk = i_item_sk
| fk/pk conjuncts: cs_item_sk = i_item_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1 row-size=20B cardinality=4.50M cost=1968701
+| tuple-ids=0,1 row-size=20B cardinality=223.54K(filtered from 4.50M) cost=872345
| in pipelines: 00(GETNEXT), 01(OPEN)
|
|--F10:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=16.52MB mem-reservation=16.50MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | Per-Instance Resources: mem-estimate=2.96MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[865]
| JOIN BUILD
| | join-table-id=03 plan-id=04 cohort-id=01
| | build expressions: i_item_sk
| | runtime filters: RF004[bloom] <- i_item_sk, RF005[min_max] <- i_item_sk
-| | mem-estimate=15.50MB mem-reservation=15.50MB spill-buffer=64.00KB thread-reservation=0 cost=375
+| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=375
| |
| 11:EXCHANGE [BROADCAST]
| | mem-estimate=20.51KB mem-reservation=0B thread-reservation=0
@@ -580,8 +580,8 @@ max-parallelism=80 segment-costs=[765509219]
table: rows=4.32G size=280.96GB
partitions: 1831/1831 rows=4.32G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=21.52M
+ extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=91(filtered from 1831)
mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
- tuple-ids=0 row-size=12B cardinality=4.50M(filtered from 4.32G) cost=746509474
+ tuple-ids=0 row-size=12B cardinality=223.54K(filtered from 4.32G) cost=37101235
in pipelines: 00(GETNEXT)
====
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q37.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q37.test
index 171741ee3..b81da4f35 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q37.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q37.test
@@ -18,10 +18,10 @@ select i_item_id
# end query 37 in stream 0 using template query37.tpl
---- PLAN
Max Per-Host Resource Reservation: Memory=73.19MB Threads=1
-Per-Host Resource Estimates: Memory=61.26GB
+Per-Host Resource Estimates: Memory=15.46GB
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=61.26GB mem-reservation=73.19MB thread-reservation=1 runtime-filters-memory=3.00MB
-| max-parallelism=1 segment-costs=[11210514257, 28288149441, 300]
+| Per-Instance Resources: mem-estimate=15.46GB mem-reservation=73.19MB thread-reservation=1 runtime-filters-memory=3.00MB
+| max-parallelism=1 segment-costs=[3569042618, 6158446489, 300]
PLAN-ROOT SINK
| output exprs: i_item_id, i_item_desc, i_current_price
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=300
@@ -29,13 +29,13 @@ PLAN-ROOT SINK
08:TOP-N [LIMIT=100]
| order by: i_item_id ASC
| mem-estimate=14.12KB mem-reservation=0B thread-reservation=0
-| tuple-ids=5 row-size=145B cardinality=100 cost=28288149441
+| tuple-ids=5 row-size=145B cardinality=100 cost=6158446489
| in pipelines: 08(GETNEXT), 07(OPEN)
|
07:AGGREGATE [FINALIZE]
| group by: i_item_id, i_item_desc, i_current_price
-| mem-estimate=60.14GB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=4 row-size=145B cardinality=1.64G cost=9675101515
+| mem-estimate=14.34GB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=4 row-size=145B cardinality=383.09M cost=2261230622
| in pipelines: 07(GETNEXT), 01(OPEN)
|
06:HASH JOIN [INNER JOIN]
@@ -43,7 +43,7 @@ PLAN-ROOT SINK
| fk/pk conjuncts: inv_date_sk = d_date_sk
| runtime filters: RF000[bloom] <- d_date_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=1,3,0,2 row-size=177B cardinality=1.64G cost=717452487
+| tuple-ids=1,3,0,2 row-size=177B cardinality=383.09M cost=489851741
| in pipelines: 01(GETNEXT), 02(OPEN)
|
|--02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
@@ -56,7 +56,7 @@ PLAN-ROOT SINK
| parquet statistics predicates: d_date <= DATE '2001-03-21', d_date >= DATE '2001-01-20'
| parquet dictionary predicates: d_date <= DATE '2001-03-21', d_date >= DATE '2001-01-20'
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
+| tuple-ids=2 row-size=8B cardinality=61 cost=12520
| in pipelines: 02(GETNEXT)
|
05:HASH JOIN [INNER JOIN]
@@ -116,7 +116,7 @@ PLAN-ROOT SINK
in pipelines: 01(GETNEXT)
---- DISTRIBUTEDPLAN
Max Per-Host Resource Reservation: Memory=1.05GB Threads=49
-Per-Host Resource Estimates: Memory=21.65GB
+Per-Host Resource Estimates: Memory=11.97GB
F06:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=5.70MB mem-reservation=4.00MB thread-reservation=1
| max-parallelism=1 segment-costs=[354] cpu-comparison-result=171 [max(1 (self) vs 171 (sum children))]
@@ -132,32 +132,32 @@ PLAN-ROOT SINK
| in pipelines: 08(GETNEXT)
|
F05:PLAN FRAGMENT [HASH(i_item_id,i_item_desc,i_current_price)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=921.40MB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=2830 segment-costs=[10947485353, 28288149441, 410] cpu-comparison-result=171 [max(120 (self) vs 171 (sum children))]
+Per-Instance Resources: mem-estimate=504.19MB mem-reservation=34.00MB thread-reservation=1
+max-parallelism=620 segment-costs=[2558607687, 6158446489, 410] cpu-comparison-result=171 [max(120 (self) vs 171 (sum children))]
08:TOP-N [LIMIT=100]
| order by: i_item_id ASC
| mem-estimate=14.12KB mem-reservation=0B thread-reservation=0
-| tuple-ids=5 row-size=145B cardinality=100 cost=28288149441
+| tuple-ids=5 row-size=145B cardinality=100 cost=6158446489
| in pipelines: 08(GETNEXT), 14(OPEN)
|
14:AGGREGATE [FINALIZE]
| group by: i_item_id, i_item_desc, i_current_price
-| mem-estimate=893.99MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=4 row-size=145B cardinality=1.64G cost=9675101515
+| mem-estimate=476.77MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=4 row-size=145B cardinality=383.09M cost=2261230622
| in pipelines: 14(GETNEXT), 01(OPEN)
|
13:EXCHANGE [HASH(i_item_id,i_item_desc,i_current_price)]
| mem-estimate=27.41MB mem-reservation=0B thread-reservation=0
-| tuple-ids=4 row-size=145B cardinality=1.64G cost=1272383838
+| tuple-ids=4 row-size=145B cardinality=383.09M cost=297377065
| in pipelines: 01(GETNEXT)
|
F03:PLAN FRAGMENT [HASH(inv_item_sk)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=846.77MB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=1620 segment-costs=[10733255133, 16108355504] cpu-comparison-result=171 [max(160 (self) vs 171 (sum children))]
+Per-Instance Resources: mem-estimate=437.86MB mem-reservation=34.00MB thread-reservation=1
+max-parallelism=380 segment-costs=[3091790738, 3764788068] cpu-comparison-result=171 [max(160 (self) vs 171 (sum children))]
07:AGGREGATE [STREAMING]
| group by: i_item_id, i_item_desc, i_current_price
-| mem-estimate=766.49MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=4 row-size=145B cardinality=1.64G cost=9675101515
+| mem-estimate=357.58MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=4 row-size=145B cardinality=383.09M cost=2261230622
| in pipelines: 01(GETNEXT)
|
06:HASH JOIN [INNER JOIN, BROADCAST]
@@ -165,26 +165,26 @@ max-parallelism=1620 segment-costs=[10733255133, 16108355504] cpu-comparison-res
| hash predicates: inv_date_sk = d_date_sk
| fk/pk conjuncts: inv_date_sk = d_date_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=1,3,0,2 row-size=177B cardinality=1.64G cost=717445182
+| tuple-ids=1,3,0,2 row-size=177B cardinality=383.09M cost=489851680
| in pipelines: 01(GETNEXT), 02(OPEN)
|
|--F07:PLAN FRAGMENT [HASH(inv_item_sk)] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
+| | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[141]
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: d_date_sk
| | runtime filters: RF000[bloom] <- d_date_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=61
| |
| 12:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=2 row-size=8B cardinality=7.30K cost=9700
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=2 row-size=8B cardinality=61 cost=80
| | in pipelines: 02(GETNEXT)
| |
| F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
+| max-parallelism=1 segment-costs=[12522]
| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| HDFS partitions=1/1 files=1 size=2.17MB
| predicates: d_date <= DATE '2001-03-21', d_date >= DATE '2001-01-20'
@@ -195,7 +195,7 @@ max-parallelism=1620 segment-costs=[10733255133, 16108355504] cpu-comparison-res
| parquet statistics predicates: d_date <= DATE '2001-03-21', d_date >= DATE '2001-01-20'
| parquet dictionary predicates: d_date <= DATE '2001-03-21', d_date >= DATE '2001-01-20'
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
+| tuple-ids=2 row-size=8B cardinality=61 cost=12520
| in pipelines: 02(GETNEXT)
|
05:HASH JOIN [INNER JOIN, PARTITIONED]
@@ -208,7 +208,7 @@ max-parallelism=1620 segment-costs=[10733255133, 16108355504] cpu-comparison-res
|
|--F08:PLAN FRAGMENT [HASH(inv_item_sk)] hosts=10 instances=120
| | Per-Instance Resources: mem-estimate=34.43MB mem-reservation=18.00MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=1620 segment-costs=[10666348] cpu-comparison-result=160 [max(160 (self) vs 14 (sum children))]
+| | max-parallelism=380 segment-costs=[10666348] cpu-comparison-result=160 [max(160 (self) vs 14 (sum children))]
| JOIN BUILD
| | join-table-id=01 plan-id=02 cohort-id=01
| | build expressions: i_item_sk
@@ -299,7 +299,7 @@ max-parallelism=40 segment-costs=[369807768]
in pipelines: 01(GETNEXT)
---- PARALLELPLANS
Max Per-Host Resource Reservation: Memory=1.05GB Threads=49
-Per-Host Resource Estimates: Memory=21.65GB
+Per-Host Resource Estimates: Memory=11.97GB
F06:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=5.70MB mem-reservation=4.00MB thread-reservation=1
| max-parallelism=1 segment-costs=[354] cpu-comparison-result=171 [max(1 (self) vs 171 (sum children))]
@@ -315,32 +315,32 @@ PLAN-ROOT SINK
| in pipelines: 08(GETNEXT)
|
F05:PLAN FRAGMENT [HASH(i_item_id,i_item_desc,i_current_price)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=921.40MB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=2830 segment-costs=[10947485353, 28288149441, 410] cpu-comparison-result=171 [max(120 (self) vs 171 (sum children))]
+Per-Instance Resources: mem-estimate=504.19MB mem-reservation=34.00MB thread-reservation=1
+max-parallelism=620 segment-costs=[2558607687, 6158446489, 410] cpu-comparison-result=171 [max(120 (self) vs 171 (sum children))]
08:TOP-N [LIMIT=100]
| order by: i_item_id ASC
| mem-estimate=14.12KB mem-reservation=0B thread-reservation=0
-| tuple-ids=5 row-size=145B cardinality=100 cost=28288149441
+| tuple-ids=5 row-size=145B cardinality=100 cost=6158446489
| in pipelines: 08(GETNEXT), 14(OPEN)
|
14:AGGREGATE [FINALIZE]
| group by: i_item_id, i_item_desc, i_current_price
-| mem-estimate=893.99MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=4 row-size=145B cardinality=1.64G cost=9675101515
+| mem-estimate=476.77MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=4 row-size=145B cardinality=383.09M cost=2261230622
| in pipelines: 14(GETNEXT), 01(OPEN)
|
13:EXCHANGE [HASH(i_item_id,i_item_desc,i_current_price)]
| mem-estimate=27.41MB mem-reservation=0B thread-reservation=0
-| tuple-ids=4 row-size=145B cardinality=1.64G cost=1272383838
+| tuple-ids=4 row-size=145B cardinality=383.09M cost=297377065
| in pipelines: 01(GETNEXT)
|
F03:PLAN FRAGMENT [HASH(inv_item_sk)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=846.77MB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=1620 segment-costs=[10733255133, 16108355504] cpu-comparison-result=171 [max(160 (self) vs 171 (sum children))]
+Per-Instance Resources: mem-estimate=437.86MB mem-reservation=34.00MB thread-reservation=1
+max-parallelism=380 segment-costs=[3091790738, 3764788068] cpu-comparison-result=171 [max(160 (self) vs 171 (sum children))]
07:AGGREGATE [STREAMING]
| group by: i_item_id, i_item_desc, i_current_price
-| mem-estimate=766.49MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=4 row-size=145B cardinality=1.64G cost=9675101515
+| mem-estimate=357.58MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=4 row-size=145B cardinality=383.09M cost=2261230622
| in pipelines: 01(GETNEXT)
|
06:HASH JOIN [INNER JOIN, BROADCAST]
@@ -348,26 +348,26 @@ max-parallelism=1620 segment-costs=[10733255133, 16108355504] cpu-comparison-res
| hash predicates: inv_date_sk = d_date_sk
| fk/pk conjuncts: inv_date_sk = d_date_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=1,3,0,2 row-size=177B cardinality=1.64G cost=717445182
+| tuple-ids=1,3,0,2 row-size=177B cardinality=383.09M cost=489851680
| in pipelines: 01(GETNEXT), 02(OPEN)
|
|--F07:PLAN FRAGMENT [HASH(inv_item_sk)] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
+| | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[141]
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: d_date_sk
| | runtime filters: RF000[bloom] <- d_date_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=61
| |
| 12:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=2 row-size=8B cardinality=7.30K cost=9700
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=2 row-size=8B cardinality=61 cost=80
| | in pipelines: 02(GETNEXT)
| |
| F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
+| max-parallelism=1 segment-costs=[12522]
| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| HDFS partitions=1/1 files=1 size=2.17MB
| predicates: d_date <= DATE '2001-03-21', d_date >= DATE '2001-01-20'
@@ -378,7 +378,7 @@ max-parallelism=1620 segment-costs=[10733255133, 16108355504] cpu-comparison-res
| parquet statistics predicates: d_date <= DATE '2001-03-21', d_date >= DATE '2001-01-20'
| parquet dictionary predicates: d_date <= DATE '2001-03-21', d_date >= DATE '2001-01-20'
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
+| tuple-ids=2 row-size=8B cardinality=61 cost=12520
| in pipelines: 02(GETNEXT)
|
05:HASH JOIN [INNER JOIN, PARTITIONED]
@@ -391,7 +391,7 @@ max-parallelism=1620 segment-costs=[10733255133, 16108355504] cpu-comparison-res
|
|--F08:PLAN FRAGMENT [HASH(inv_item_sk)] hosts=10 instances=120
| | Per-Instance Resources: mem-estimate=34.43MB mem-reservation=18.00MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=1620 segment-costs=[10666348] cpu-comparison-result=160 [max(160 (self) vs 14 (sum children))]
+| | max-parallelism=380 segment-costs=[10666348] cpu-comparison-result=160 [max(160 (self) vs 14 (sum children))]
| JOIN BUILD
| | join-table-id=01 plan-id=02 cohort-id=01
| | build expressions: i_item_sk
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q40.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q40.test
index c2a1b34e0..536d7838b 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q40.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q40.test
@@ -32,7 +32,7 @@ Max Per-Host Resource Reservation: Memory=78.75MB Threads=1
Per-Host Resource Estimates: Memory=18.66GB
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=18.66GB mem-reservation=78.75MB thread-reservation=1 runtime-filters-memory=3.00MB
-| max-parallelism=1 segment-costs=[4119787190, 28837420, 400]
+| max-parallelism=1 segment-costs=[1541715036, 28837420, 400]
PLAN-ROOT SINK
| output exprs: w_state, i_item_id, sum(CASE WHEN d_date < DATE '1999-02-02' THEN cs_sales_price - coalesce(cr_refunded_cash, 0) ELSE 0 END), sum(CASE WHEN d_date >= DATE '1999-02-02' THEN cs_sales_price - coalesce(cr_refunded_cash, 0) ELSE 0 END)
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=400
@@ -47,36 +47,15 @@ PLAN-ROOT SINK
| output: sum(CASE WHEN d_date < DATE '1999-02-02' THEN cs_sales_price - coalesce(cr_refunded_cash, CAST(0 AS DECIMAL(7,2))) ELSE CAST(0 AS DECIMAL(8,2)) END), sum(CASE WHEN d_date >= DATE '1999-02-02' THEN cs_sales_price - coalesce(cr_refunded_cash, CAST(0 AS DECIMAL(7,2))) ELSE CAST(0 AS DECIMAL(8,2)) END)
| group by: w_state, i_item_id
| mem-estimate=197.86MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=5 row-size=74B cardinality=2.41M cost=634475343
+| tuple-ids=5 row-size=74B cardinality=2.41M cost=31698329
| in pipelines: 09(GETNEXT), 00(OPEN)
|
08:HASH JOIN [INNER JOIN]
-| hash predicates: cs_sold_date_sk = d_date_sk
-| fk/pk conjuncts: cs_sold_date_sk = d_date_sk
-| runtime filters: RF000[bloom] <- d_date_sk
-| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1N,3,2,4 row-size=102B cardinality=431.79M cost=189002620
-| in pipelines: 00(GETNEXT), 04(OPEN)
-|
-|--04:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
-| HDFS partitions=1/1 files=1 size=2.17MB
-| predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
-| stored statistics:
-| table: rows=73.05K size=2.17MB
-| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=73.05K
-| parquet statistics predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
-| parquet dictionary predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
-| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=4 row-size=8B cardinality=7.30K cost=12520
-| in pipelines: 04(GETNEXT)
-|
-07:HASH JOIN [INNER JOIN]
| hash predicates: cs_warehouse_sk = w_warehouse_sk
| fk/pk conjuncts: cs_warehouse_sk = w_warehouse_sk
-| runtime filters: RF002[bloom] <- w_warehouse_sk, RF003[min_max] <- w_warehouse_sk
+| runtime filters: RF000[bloom] <- w_warehouse_sk, RF001[min_max] <- w_warehouse_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1N,3,2 row-size=94B cardinality=431.79M cost=188995337
+| tuple-ids=0,1N,4,3,2 row-size=102B cardinality=14.39M cost=6296425
| in pipelines: 00(GETNEXT), 02(OPEN)
|
|--02:SCAN HDFS [tpcds_partitioned_parquet_snap.warehouse]
@@ -89,12 +68,12 @@ PLAN-ROOT SINK
| tuple-ids=2 row-size=18B cardinality=22 cost=1
| in pipelines: 02(GETNEXT)
|
-06:HASH JOIN [INNER JOIN]
+07:HASH JOIN [INNER JOIN]
| hash predicates: cs_item_sk = i_item_sk
| fk/pk conjuncts: cs_item_sk = i_item_sk
-| runtime filters: RF004[bloom] <- i_item_sk, RF005[min_max] <- i_item_sk
+| runtime filters: RF002[bloom] <- i_item_sk, RF003[min_max] <- i_item_sk
| mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
-| tuple-ids=0,1N,3 row-size=76B cardinality=431.79M cost=189031315
+| tuple-ids=0,1N,4,3 row-size=84B cardinality=14.39M cost=39559092
| in pipelines: 00(GETNEXT), 03(OPEN)
|
|--03:SCAN HDFS [tpcds_partitioned_parquet_snap.item]
@@ -110,16 +89,37 @@ PLAN-ROOT SINK
| tuple-ids=3 row-size=36B cardinality=36.00K cost=144648
| in pipelines: 03(GETNEXT)
|
+06:HASH JOIN [INNER JOIN]
+| hash predicates: cs_sold_date_sk = d_date_sk
+| fk/pk conjuncts: cs_sold_date_sk = d_date_sk
+| runtime filters: RF004[bloom] <- d_date_sk
+| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,1N,4 row-size=48B cardinality=143.92M cost=62995575
+| in pipelines: 00(GETNEXT), 04(OPEN)
+|
+|--04:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
+| HDFS partitions=1/1 files=1 size=2.17MB
+| predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
+| stored statistics:
+| table: rows=73.05K size=2.17MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=73.05K
+| parquet statistics predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
+| parquet dictionary predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
+| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| tuple-ids=4 row-size=8B cardinality=61 cost=12520
+| in pipelines: 04(GETNEXT)
+|
05:HASH JOIN [LEFT OUTER JOIN]
| hash predicates: cs_item_sk = cr_item_sk, cs_order_number = cr_order_number
| fk/pk conjuncts: cs_item_sk = cr_item_sk, cs_order_number = cr_order_number
| mem-estimate=18.44GB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,1N row-size=40B cardinality=431.79M(filtered from 4.32G) cost=1325569743
+| tuple-ids=0,1N row-size=40B cardinality=143.92M(filtered from 4.32G) cost=1251731614
| in pipelines: 00(GETNEXT), 01(OPEN)
|
|--01:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_returns]
| HDFS partitions=2060/2060 files=2060 size=30.47MB
-| runtime filters: RF005[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk
+| runtime filters: RF003[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk, RF002[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk
| stored statistics:
| table: rows=432.02M size=32.77GB
| partitions: 2060/2060 rows=432.02M
@@ -131,21 +131,21 @@ PLAN-ROOT SINK
|
00:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales]
HDFS partitions=1831/1831 files=1831 size=150.82MB
- runtime filters: RF003[min_max] -> cs_warehouse_sk, RF005[min_max] -> cs_item_sk, RF000[bloom] -> cs_sold_date_sk, RF002[bloom] -> cs_warehouse_sk, RF004[bloom] -> cs_item_sk
+ runtime filters: RF001[min_max] -> cs_warehouse_sk, RF003[min_max] -> cs_item_sk, RF000[bloom] -> cs_warehouse_sk, RF002[bloom] -> cs_item_sk, RF004[bloom] -> cs_sold_date_sk
stored statistics:
table: rows=4.32G size=280.96GB
partitions: 1831/1831 rows=4.32G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=21.52M
+ extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=62(filtered from 1831)
mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
- tuple-ids=0 row-size=24B cardinality=431.79M(filtered from 4.32G) cost=1493018949
+ tuple-ids=0 row-size=24B cardinality=143.92M(filtered from 4.32G) cost=49740118
in pipelines: 00(GETNEXT)
---- DISTRIBUTEDPLAN
-Max Per-Host Resource Reservation: Memory=955.77MB Threads=52
-Per-Host Resource Estimates: Memory=5.22GB
+Max Per-Host Resource Reservation: Memory=686.02MB Threads=42
+Per-Host Resource Estimates: Memory=2.79GB
F07:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=4.08MB mem-reservation=4.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[440] cpu-comparison-result=240 [max(1 (self) vs 240 (sum children))]
+| max-parallelism=1 segment-costs=[440] cpu-comparison-result=226 [max(1 (self) vs 226 (sum children))]
PLAN-ROOT SINK
| output exprs: w_state, i_item_id, sum(CASE WHEN d_date < DATE '1999-02-02' THEN cs_sales_price - coalesce(cr_refunded_cash, 0) ELSE 0 END), sum(CASE WHEN d_date >= DATE '1999-02-02' THEN cs_sales_price - coalesce(cr_refunded_cash, 0) ELSE 0 END)
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=400
@@ -158,8 +158,8 @@ PLAN-ROOT SINK
| in pipelines: 10(GETNEXT)
|
F06:PLAN FRAGMENT [HASH(w_state,i_item_id)] hosts=10 instances=10 (adjusted from 120)
-Per-Instance Resources: mem-estimate=53.14MB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=10 segment-costs=[15497645, 28837420, 225] cpu-comparison-result=240 [max(10 (self) vs 240 (sum children))]
+Per-Instance Resources: mem-estimate=52.38MB mem-reservation=34.00MB thread-reservation=1
+max-parallelism=10 segment-costs=[15497645, 28837420, 225] cpu-comparison-result=226 [max(10 (self) vs 226 (sum children))]
10:TOP-N [LIMIT=100]
| order by: w_state ASC, i_item_id ASC
| mem-estimate=7.23KB mem-reservation=0B thread-reservation=0
@@ -174,81 +174,43 @@ max-parallelism=10 segment-costs=[15497645, 28837420, 225] cpu-comparison-result
| in pipelines: 17(GETNEXT), 00(OPEN)
|
16:EXCHANGE [HASH(w_state,i_item_id)]
-| mem-estimate=19.14MB mem-reservation=0B thread-reservation=0
+| mem-estimate=18.38MB mem-reservation=0B thread-reservation=0
| tuple-ids=5 row-size=74B cardinality=2.41M cost=1089208
| in pipelines: 00(GETNEXT)
|
-F02:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=214.19MB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=350 segment-costs=[3490683130, 12739523] cpu-comparison-result=240 [max(240 (self) vs 236 (sum children))]
+F02:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=110 (adjusted from 120)
+Per-Instance Resources: mem-estimate=31.14MB mem-reservation=17.00MB thread-reservation=1
+max-parallelism=110 segment-costs=[1046325850, 12739523] cpu-comparison-result=226 [max(150 (self) vs 226 (sum children))]
09:AGGREGATE [STREAMING]
| output: sum(CASE WHEN d_date < DATE '1999-02-02' THEN cs_sales_price - coalesce(cr_refunded_cash, CAST(0 AS DECIMAL(7,2))) ELSE CAST(0 AS DECIMAL(8,2)) END), sum(CASE WHEN d_date >= DATE '1999-02-02' THEN cs_sales_price - coalesce(cr_refunded_cash, CAST(0 AS DECIMAL(7,2))) ELSE CAST(0 AS DECIMAL(8,2)) END)
| group by: w_state, i_item_id
-| mem-estimate=197.86MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=5 row-size=74B cardinality=2.41M cost=1934502527
+| mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB thread-reservation=0
+| tuple-ids=5 row-size=74B cardinality=2.41M cost=85917063
| in pipelines: 00(GETNEXT)
|
08:HASH JOIN [INNER JOIN, BROADCAST]
| hash-table-id=00
-| hash predicates: cs_sold_date_sk = d_date_sk
-| fk/pk conjuncts: cs_sold_date_sk = d_date_sk
-| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1N,3,2,4 row-size=102B cardinality=431.79M cost=188995315
-| in pipelines: 00(GETNEXT), 04(OPEN)
-|
-|--F08:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
-| JOIN BUILD
-| | join-table-id=00 plan-id=01 cohort-id=01
-| | build expressions: d_date_sk
-| | runtime filters: RF000[bloom] <- d_date_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
-| |
-| 15:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=4 row-size=8B cardinality=7.30K cost=9700
-| | in pipelines: 04(GETNEXT)
-| |
-| F05:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
-| 04:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| HDFS partitions=1/1 files=1 size=2.17MB
-| predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
-| stored statistics:
-| table: rows=73.05K size=2.17MB
-| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=73.05K
-| parquet statistics predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
-| parquet dictionary predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
-| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=4 row-size=8B cardinality=7.30K cost=12520
-| in pipelines: 04(GETNEXT)
-|
-07:HASH JOIN [INNER JOIN, BROADCAST]
-| hash-table-id=01
| hash predicates: cs_warehouse_sk = w_warehouse_sk
| fk/pk conjuncts: cs_warehouse_sk = w_warehouse_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1N,3,2 row-size=94B cardinality=431.79M cost=188995315
+| tuple-ids=0,1N,4,3,2 row-size=102B cardinality=14.39M cost=6296403
| in pipelines: 00(GETNEXT), 02(OPEN)
|
-|--F09:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+|--F08:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
+| | Per-Instance Resources: mem-estimate=22.33MB mem-reservation=22.31MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[42]
| JOIN BUILD
-| | join-table-id=01 plan-id=02 cohort-id=01
+| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: w_warehouse_sk
-| | runtime filters: RF002[bloom] <- w_warehouse_sk, RF003[min_max] <- w_warehouse_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=22
+| | runtime filters: RF000[bloom] <- w_warehouse_sk, RF001[min_max] <- w_warehouse_sk
+| | mem-estimate=21.31MB mem-reservation=21.31MB spill-buffer=64.00KB thread-reservation=0 cost=22
| |
-| 14:EXCHANGE [BROADCAST]
+| 15:EXCHANGE [BROADCAST]
| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
| | tuple-ids=2 row-size=18B cardinality=22 cost=20
| | in pipelines: 02(GETNEXT)
| |
-| F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| F05:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=16.09MB mem-reservation=16.00KB thread-reservation=1
| max-parallelism=1 segment-costs=[2]
| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.warehouse, RANDOM]
@@ -261,29 +223,29 @@ max-parallelism=350 segment-costs=[3490683130, 12739523] cpu-comparison-result=2
| tuple-ids=2 row-size=18B cardinality=22 cost=1
| in pipelines: 02(GETNEXT)
|
-06:HASH JOIN [INNER JOIN, BROADCAST]
-| hash-table-id=02
+07:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=01
| hash predicates: cs_item_sk = i_item_sk
| fk/pk conjuncts: cs_item_sk = i_item_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
-| tuple-ids=0,1N,3 row-size=76B cardinality=431.79M cost=188995315
+| tuple-ids=0,1N,4,3 row-size=84B cardinality=14.39M cost=39523092
| in pipelines: 00(GETNEXT), 03(OPEN)
|
-|--F10:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=36.89MB mem-reservation=35.50MB thread-reservation=1 runtime-filters-memory=1.00MB
+|--F09:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
+| | Per-Instance Resources: mem-estimate=34.02MB mem-reservation=32.62MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[83840]
| JOIN BUILD
-| | join-table-id=02 plan-id=03 cohort-id=01
+| | join-table-id=01 plan-id=02 cohort-id=01
| | build expressions: i_item_sk
-| | runtime filters: RF004[bloom] <- i_item_sk, RF005[min_max] <- i_item_sk
-| | mem-estimate=34.50MB mem-reservation=34.50MB spill-buffer=128.00KB thread-reservation=0 cost=36000
+| | runtime filters: RF002[bloom] <- i_item_sk, RF003[min_max] <- i_item_sk
+| | mem-estimate=31.62MB mem-reservation=31.62MB spill-buffer=128.00KB thread-reservation=0 cost=36000
| |
-| 13:EXCHANGE [BROADCAST]
+| 14:EXCHANGE [BROADCAST]
| | mem-estimate=1.39MB mem-reservation=0B thread-reservation=0
| | tuple-ids=3 row-size=36B cardinality=36.00K cost=47840
| | in pipelines: 03(GETNEXT)
| |
-| F03:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
+| F04:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
| Per-Instance Resources: mem-estimate=16.16MB mem-reservation=512.00KB thread-reservation=1
| max-parallelism=4 segment-costs=[148924]
| 03:SCAN HDFS [tpcds_partitioned_parquet_snap.item, RANDOM]
@@ -299,21 +261,59 @@ max-parallelism=350 segment-costs=[3490683130, 12739523] cpu-comparison-result=2
| tuple-ids=3 row-size=36B cardinality=36.00K cost=144648
| in pipelines: 03(GETNEXT)
|
+06:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=02
+| hash predicates: cs_sold_date_sk = d_date_sk
+| fk/pk conjuncts: cs_sold_date_sk = d_date_sk
+| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,1N,4 row-size=48B cardinality=143.92M cost=62995514
+| in pipelines: 00(GETNEXT), 04(OPEN)
+|
+|--F10:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
+| | Per-Instance Resources: mem-estimate=22.33MB mem-reservation=22.31MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[141]
+| JOIN BUILD
+| | join-table-id=02 plan-id=03 cohort-id=01
+| | build expressions: d_date_sk
+| | runtime filters: RF004[bloom] <- d_date_sk
+| | mem-estimate=21.31MB mem-reservation=21.31MB spill-buffer=64.00KB thread-reservation=0 cost=61
+| |
+| 13:EXCHANGE [BROADCAST]
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=4 row-size=8B cardinality=61 cost=80
+| | in pipelines: 04(GETNEXT)
+| |
+| F03:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| max-parallelism=1 segment-costs=[12522]
+| 04:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| HDFS partitions=1/1 files=1 size=2.17MB
+| predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
+| stored statistics:
+| table: rows=73.05K size=2.17MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=73.05K
+| parquet statistics predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
+| parquet dictionary predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
+| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| tuple-ids=4 row-size=8B cardinality=61 cost=12520
+| in pipelines: 04(GETNEXT)
+|
05:HASH JOIN [LEFT OUTER JOIN, PARTITIONED]
| hash-table-id=03
| hash predicates: cs_item_sk = cr_item_sk, cs_order_number = cr_order_number
| fk/pk conjuncts: cs_item_sk = cr_item_sk, cs_order_number = cr_order_number
| mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,1N row-size=40B cardinality=431.79M(filtered from 4.32G) cost=893552752
+| tuple-ids=0,1N row-size=40B cardinality=143.92M(filtered from 4.32G) cost=819714623
| in pipelines: 00(GETNEXT), 01(OPEN)
|
-|--F11:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=120
-| | Per-Instance Resources: mem-estimate=168.90MB mem-reservation=34.00MB thread-reservation=1
-| | max-parallelism=350 segment-costs=[511810529]
+|--F11:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=110 (adjusted from 120)
+| | Per-Instance Resources: mem-estimate=183.20MB mem-reservation=34.00MB thread-reservation=1
+| | max-parallelism=110 segment-costs=[511810529]
| JOIN BUILD
| | join-table-id=03 plan-id=04 cohort-id=01
| | build expressions: cr_item_sk, cr_order_number
-| | mem-estimate=157.33MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 cost=432016991
+| | mem-estimate=171.64MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 cost=432016991
| |
| 12:EXCHANGE [HASH(cr_item_sk,cr_order_number)]
| | mem-estimate=11.56MB mem-reservation=0B thread-reservation=0
@@ -322,11 +322,11 @@ max-parallelism=350 segment-costs=[3490683130, 12739523] cpu-comparison-result=2
| |
| F01:PLAN FRAGMENT [RANDOM] hosts=10 instances=80 (adjusted from 120)
| Per-Host Shared Resources: mem-estimate=1.00MB mem-reservation=1.00MB thread-reservation=0 runtime-filters-memory=1.00MB
-| Per-Instance Resources: mem-estimate=25.38MB mem-reservation=32.00KB thread-reservation=1
+| Per-Instance Resources: mem-estimate=24.59MB mem-reservation=32.00KB thread-reservation=1
| max-parallelism=80 segment-costs=[767262175]
| 01:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_returns, RANDOM]
| HDFS partitions=2060/2060 files=2060 size=30.47MB
-| runtime filters: RF005[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk
+| runtime filters: RF003[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk, RF002[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk
| stored statistics:
| table: rows=432.02M size=32.77GB
| partitions: 2060/2060 rows=432.02M
@@ -337,31 +337,31 @@ max-parallelism=350 segment-costs=[3490683130, 12739523] cpu-comparison-result=2
| in pipelines: 01(GETNEXT)
|
11:EXCHANGE [HASH(cs_item_sk,cs_order_number)]
-| mem-estimate=13.28MB mem-reservation=0B thread-reservation=0
-| tuple-ids=0 row-size=24B cardinality=431.79M(filtered from 4.32G) cost=95641906
+| mem-estimate=11.09MB mem-reservation=0B thread-reservation=0
+| tuple-ids=0 row-size=24B cardinality=143.92M(filtered from 4.32G) cost=31879155
| in pipelines: 00(GETNEXT)
|
-F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=40 (adjusted from 120)
Per-Host Shared Resources: mem-estimate=3.00MB mem-reservation=3.00MB thread-reservation=0 runtime-filters-memory=3.00MB
-Per-Instance Resources: mem-estimate=29.12MB mem-reservation=1.00MB thread-reservation=1
-max-parallelism=240 segment-costs=[2382855700]
+Per-Instance Resources: mem-estimate=28.03MB mem-reservation=1.00MB thread-reservation=1
+max-parallelism=40 segment-costs=[346338600]
00:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales, RANDOM]
HDFS partitions=1831/1831 files=1831 size=150.82MB
- runtime filters: RF003[min_max] -> cs_warehouse_sk, RF005[min_max] -> cs_item_sk, RF000[bloom] -> cs_sold_date_sk, RF002[bloom] -> cs_warehouse_sk, RF004[bloom] -> cs_item_sk
+ runtime filters: RF001[min_max] -> cs_warehouse_sk, RF003[min_max] -> cs_item_sk, RF000[bloom] -> cs_warehouse_sk, RF002[bloom] -> cs_item_sk, RF004[bloom] -> cs_sold_date_sk
stored statistics:
table: rows=4.32G size=280.96GB
partitions: 1831/1831 rows=4.32G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=21.52M
+ extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=62(filtered from 1831)
mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
- tuple-ids=0 row-size=24B cardinality=431.79M(filtered from 4.32G) cost=1493018949
+ tuple-ids=0 row-size=24B cardinality=143.92M(filtered from 4.32G) cost=49740118
in pipelines: 00(GETNEXT)
---- PARALLELPLANS
-Max Per-Host Resource Reservation: Memory=955.77MB Threads=52
-Per-Host Resource Estimates: Memory=5.22GB
+Max Per-Host Resource Reservation: Memory=686.02MB Threads=42
+Per-Host Resource Estimates: Memory=2.79GB
F07:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=4.08MB mem-reservation=4.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[440] cpu-comparison-result=240 [max(1 (self) vs 240 (sum children))]
+| max-parallelism=1 segment-costs=[440] cpu-comparison-result=226 [max(1 (self) vs 226 (sum children))]
PLAN-ROOT SINK
| output exprs: w_state, i_item_id, sum(CASE WHEN d_date < DATE '1999-02-02' THEN cs_sales_price - coalesce(cr_refunded_cash, 0) ELSE 0 END), sum(CASE WHEN d_date >= DATE '1999-02-02' THEN cs_sales_price - coalesce(cr_refunded_cash, 0) ELSE 0 END)
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=400
@@ -374,8 +374,8 @@ PLAN-ROOT SINK
| in pipelines: 10(GETNEXT)
|
F06:PLAN FRAGMENT [HASH(w_state,i_item_id)] hosts=10 instances=10 (adjusted from 120)
-Per-Instance Resources: mem-estimate=53.14MB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=10 segment-costs=[15497645, 28837420, 225] cpu-comparison-result=240 [max(10 (self) vs 240 (sum children))]
+Per-Instance Resources: mem-estimate=52.38MB mem-reservation=34.00MB thread-reservation=1
+max-parallelism=10 segment-costs=[15497645, 28837420, 225] cpu-comparison-result=226 [max(10 (self) vs 226 (sum children))]
10:TOP-N [LIMIT=100]
| order by: w_state ASC, i_item_id ASC
| mem-estimate=7.23KB mem-reservation=0B thread-reservation=0
@@ -390,81 +390,43 @@ max-parallelism=10 segment-costs=[15497645, 28837420, 225] cpu-comparison-result
| in pipelines: 17(GETNEXT), 00(OPEN)
|
16:EXCHANGE [HASH(w_state,i_item_id)]
-| mem-estimate=19.14MB mem-reservation=0B thread-reservation=0
+| mem-estimate=18.38MB mem-reservation=0B thread-reservation=0
| tuple-ids=5 row-size=74B cardinality=2.41M cost=1089208
| in pipelines: 00(GETNEXT)
|
-F02:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=214.19MB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=350 segment-costs=[3490683130, 12739523] cpu-comparison-result=240 [max(240 (self) vs 236 (sum children))]
+F02:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=110 (adjusted from 120)
+Per-Instance Resources: mem-estimate=31.14MB mem-reservation=17.00MB thread-reservation=1
+max-parallelism=110 segment-costs=[1046325850, 12739523] cpu-comparison-result=226 [max(150 (self) vs 226 (sum children))]
09:AGGREGATE [STREAMING]
| output: sum(CASE WHEN d_date < DATE '1999-02-02' THEN cs_sales_price - coalesce(cr_refunded_cash, CAST(0 AS DECIMAL(7,2))) ELSE CAST(0 AS DECIMAL(8,2)) END), sum(CASE WHEN d_date >= DATE '1999-02-02' THEN cs_sales_price - coalesce(cr_refunded_cash, CAST(0 AS DECIMAL(7,2))) ELSE CAST(0 AS DECIMAL(8,2)) END)
| group by: w_state, i_item_id
-| mem-estimate=197.86MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=5 row-size=74B cardinality=2.41M cost=1934502527
+| mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB thread-reservation=0
+| tuple-ids=5 row-size=74B cardinality=2.41M cost=85917063
| in pipelines: 00(GETNEXT)
|
08:HASH JOIN [INNER JOIN, BROADCAST]
| hash-table-id=00
-| hash predicates: cs_sold_date_sk = d_date_sk
-| fk/pk conjuncts: cs_sold_date_sk = d_date_sk
-| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1N,3,2,4 row-size=102B cardinality=431.79M cost=188995315
-| in pipelines: 00(GETNEXT), 04(OPEN)
-|
-|--F08:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
-| JOIN BUILD
-| | join-table-id=00 plan-id=01 cohort-id=01
-| | build expressions: d_date_sk
-| | runtime filters: RF000[bloom] <- d_date_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
-| |
-| 15:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=4 row-size=8B cardinality=7.30K cost=9700
-| | in pipelines: 04(GETNEXT)
-| |
-| F05:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
-| 04:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| HDFS partitions=1/1 files=1 size=2.17MB
-| predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
-| stored statistics:
-| table: rows=73.05K size=2.17MB
-| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=73.05K
-| parquet statistics predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
-| parquet dictionary predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
-| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=4 row-size=8B cardinality=7.30K cost=12520
-| in pipelines: 04(GETNEXT)
-|
-07:HASH JOIN [INNER JOIN, BROADCAST]
-| hash-table-id=01
| hash predicates: cs_warehouse_sk = w_warehouse_sk
| fk/pk conjuncts: cs_warehouse_sk = w_warehouse_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1N,3,2 row-size=94B cardinality=431.79M cost=188995315
+| tuple-ids=0,1N,4,3,2 row-size=102B cardinality=14.39M cost=6296403
| in pipelines: 00(GETNEXT), 02(OPEN)
|
-|--F09:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+|--F08:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
+| | Per-Instance Resources: mem-estimate=22.33MB mem-reservation=22.31MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[42]
| JOIN BUILD
-| | join-table-id=01 plan-id=02 cohort-id=01
+| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: w_warehouse_sk
-| | runtime filters: RF002[bloom] <- w_warehouse_sk, RF003[min_max] <- w_warehouse_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=22
+| | runtime filters: RF000[bloom] <- w_warehouse_sk, RF001[min_max] <- w_warehouse_sk
+| | mem-estimate=21.31MB mem-reservation=21.31MB spill-buffer=64.00KB thread-reservation=0 cost=22
| |
-| 14:EXCHANGE [BROADCAST]
+| 15:EXCHANGE [BROADCAST]
| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
| | tuple-ids=2 row-size=18B cardinality=22 cost=20
| | in pipelines: 02(GETNEXT)
| |
-| F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| F05:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=16.09MB mem-reservation=16.00KB thread-reservation=1
| max-parallelism=1 segment-costs=[2]
| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.warehouse, RANDOM]
@@ -477,29 +439,29 @@ max-parallelism=350 segment-costs=[3490683130, 12739523] cpu-comparison-result=2
| tuple-ids=2 row-size=18B cardinality=22 cost=1
| in pipelines: 02(GETNEXT)
|
-06:HASH JOIN [INNER JOIN, BROADCAST]
-| hash-table-id=02
+07:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=01
| hash predicates: cs_item_sk = i_item_sk
| fk/pk conjuncts: cs_item_sk = i_item_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
-| tuple-ids=0,1N,3 row-size=76B cardinality=431.79M cost=188995315
+| tuple-ids=0,1N,4,3 row-size=84B cardinality=14.39M cost=39523092
| in pipelines: 00(GETNEXT), 03(OPEN)
|
-|--F10:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=36.89MB mem-reservation=35.50MB thread-reservation=1 runtime-filters-memory=1.00MB
+|--F09:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
+| | Per-Instance Resources: mem-estimate=34.02MB mem-reservation=32.62MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[83840]
| JOIN BUILD
-| | join-table-id=02 plan-id=03 cohort-id=01
+| | join-table-id=01 plan-id=02 cohort-id=01
| | build expressions: i_item_sk
-| | runtime filters: RF004[bloom] <- i_item_sk, RF005[min_max] <- i_item_sk
-| | mem-estimate=34.50MB mem-reservation=34.50MB spill-buffer=128.00KB thread-reservation=0 cost=36000
+| | runtime filters: RF002[bloom] <- i_item_sk, RF003[min_max] <- i_item_sk
+| | mem-estimate=31.62MB mem-reservation=31.62MB spill-buffer=128.00KB thread-reservation=0 cost=36000
| |
-| 13:EXCHANGE [BROADCAST]
+| 14:EXCHANGE [BROADCAST]
| | mem-estimate=1.39MB mem-reservation=0B thread-reservation=0
| | tuple-ids=3 row-size=36B cardinality=36.00K cost=47840
| | in pipelines: 03(GETNEXT)
| |
-| F03:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
+| F04:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
| Per-Instance Resources: mem-estimate=16.16MB mem-reservation=512.00KB thread-reservation=1
| max-parallelism=4 segment-costs=[148924]
| 03:SCAN HDFS [tpcds_partitioned_parquet_snap.item, RANDOM]
@@ -515,21 +477,59 @@ max-parallelism=350 segment-costs=[3490683130, 12739523] cpu-comparison-result=2
| tuple-ids=3 row-size=36B cardinality=36.00K cost=144648
| in pipelines: 03(GETNEXT)
|
+06:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=02
+| hash predicates: cs_sold_date_sk = d_date_sk
+| fk/pk conjuncts: cs_sold_date_sk = d_date_sk
+| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,1N,4 row-size=48B cardinality=143.92M cost=62995514
+| in pipelines: 00(GETNEXT), 04(OPEN)
+|
+|--F10:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
+| | Per-Instance Resources: mem-estimate=22.33MB mem-reservation=22.31MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[141]
+| JOIN BUILD
+| | join-table-id=02 plan-id=03 cohort-id=01
+| | build expressions: d_date_sk
+| | runtime filters: RF004[bloom] <- d_date_sk
+| | mem-estimate=21.31MB mem-reservation=21.31MB spill-buffer=64.00KB thread-reservation=0 cost=61
+| |
+| 13:EXCHANGE [BROADCAST]
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=4 row-size=8B cardinality=61 cost=80
+| | in pipelines: 04(GETNEXT)
+| |
+| F03:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| max-parallelism=1 segment-costs=[12522]
+| 04:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| HDFS partitions=1/1 files=1 size=2.17MB
+| predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
+| stored statistics:
+| table: rows=73.05K size=2.17MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=73.05K
+| parquet statistics predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
+| parquet dictionary predicates: d_date <= DATE '1999-03-04', d_date >= DATE '1999-01-03'
+| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| tuple-ids=4 row-size=8B cardinality=61 cost=12520
+| in pipelines: 04(GETNEXT)
+|
05:HASH JOIN [LEFT OUTER JOIN, PARTITIONED]
| hash-table-id=03
| hash predicates: cs_item_sk = cr_item_sk, cs_order_number = cr_order_number
| fk/pk conjuncts: cs_item_sk = cr_item_sk, cs_order_number = cr_order_number
| mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,1N row-size=40B cardinality=431.79M(filtered from 4.32G) cost=893552752
+| tuple-ids=0,1N row-size=40B cardinality=143.92M(filtered from 4.32G) cost=819714623
| in pipelines: 00(GETNEXT), 01(OPEN)
|
-|--F11:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=120
-| | Per-Instance Resources: mem-estimate=168.90MB mem-reservation=34.00MB thread-reservation=1
-| | max-parallelism=350 segment-costs=[511810529]
+|--F11:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=110 (adjusted from 120)
+| | Per-Instance Resources: mem-estimate=183.20MB mem-reservation=34.00MB thread-reservation=1
+| | max-parallelism=110 segment-costs=[511810529]
| JOIN BUILD
| | join-table-id=03 plan-id=04 cohort-id=01
| | build expressions: cr_item_sk, cr_order_number
-| | mem-estimate=157.33MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 cost=432016991
+| | mem-estimate=171.64MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 cost=432016991
| |
| 12:EXCHANGE [HASH(cr_item_sk,cr_order_number)]
| | mem-estimate=11.56MB mem-reservation=0B thread-reservation=0
@@ -538,11 +538,11 @@ max-parallelism=350 segment-costs=[3490683130, 12739523] cpu-comparison-result=2
| |
| F01:PLAN FRAGMENT [RANDOM] hosts=10 instances=80 (adjusted from 120)
| Per-Host Shared Resources: mem-estimate=1.00MB mem-reservation=1.00MB thread-reservation=0 runtime-filters-memory=1.00MB
-| Per-Instance Resources: mem-estimate=25.38MB mem-reservation=32.00KB thread-reservation=1
+| Per-Instance Resources: mem-estimate=24.59MB mem-reservation=32.00KB thread-reservation=1
| max-parallelism=80 segment-costs=[767262175]
| 01:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_returns, RANDOM]
| HDFS partitions=2060/2060 files=2060 size=30.47MB
-| runtime filters: RF005[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk
+| runtime filters: RF003[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk, RF002[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk
| stored statistics:
| table: rows=432.02M size=32.77GB
| partitions: 2060/2060 rows=432.02M
@@ -553,23 +553,23 @@ max-parallelism=350 segment-costs=[3490683130, 12739523] cpu-comparison-result=2
| in pipelines: 01(GETNEXT)
|
11:EXCHANGE [HASH(cs_item_sk,cs_order_number)]
-| mem-estimate=13.28MB mem-reservation=0B thread-reservation=0
-| tuple-ids=0 row-size=24B cardinality=431.79M(filtered from 4.32G) cost=95641906
+| mem-estimate=11.09MB mem-reservation=0B thread-reservation=0
+| tuple-ids=0 row-size=24B cardinality=143.92M(filtered from 4.32G) cost=31879155
| in pipelines: 00(GETNEXT)
|
-F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=40 (adjusted from 120)
Per-Host Shared Resources: mem-estimate=3.00MB mem-reservation=3.00MB thread-reservation=0 runtime-filters-memory=3.00MB
-Per-Instance Resources: mem-estimate=29.12MB mem-reservation=1.00MB thread-reservation=1
-max-parallelism=240 segment-costs=[2382855700]
+Per-Instance Resources: mem-estimate=28.03MB mem-reservation=1.00MB thread-reservation=1
+max-parallelism=40 segment-costs=[346338600]
00:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales, RANDOM]
HDFS partitions=1831/1831 files=1831 size=150.82MB
- runtime filters: RF003[min_max] -> cs_warehouse_sk, RF005[min_max] -> cs_item_sk, RF000[bloom] -> cs_sold_date_sk, RF002[bloom] -> cs_warehouse_sk, RF004[bloom] -> cs_item_sk
+ runtime filters: RF001[min_max] -> cs_warehouse_sk, RF003[min_max] -> cs_item_sk, RF000[bloom] -> cs_warehouse_sk, RF002[bloom] -> cs_item_sk, RF004[bloom] -> cs_sold_date_sk
stored statistics:
table: rows=4.32G size=280.96GB
partitions: 1831/1831 rows=4.32G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=21.52M
+ extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=62(filtered from 1831)
mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
- tuple-ids=0 row-size=24B cardinality=431.79M(filtered from 4.32G) cost=1493018949
+ tuple-ids=0 row-size=24B cardinality=143.92M(filtered from 4.32G) cost=49740118
in pipelines: 00(GETNEXT)
====
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q77.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q77.test
index 5433e5d2e..300fea931 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q77.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q77.test
@@ -112,7 +112,7 @@ Max Per-Host Resource Reservation: Memory=36.88MB Threads=1
Per-Host Resource Estimates: Memory=144MB
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=143.75MB mem-reservation=36.88MB thread-reservation=1 runtime-filters-memory=10.00MB
-| max-parallelism=1 segment-costs=[12686041532, 1268619197, 4452292314, 445256659, 3126244429, 312662682, 74615, 43669, 52232, 500]
+| max-parallelism=1 segment-costs=[215624061, 19641382, 75392491, 6712839, 53589087, 4647418, 74615, 43669, 52232, 500]
PLAN-ROOT SINK
| output exprs: CASE valid_tid(32,30,31) WHEN 30 THEN channel WHEN 31 THEN channel WHEN 32 THEN NULL END, CASE valid_tid(32,30,31) WHEN 30 THEN id WHEN 31 THEN NULL WHEN 32 THEN NULL END, aggif(valid_tid(32,30,31) IN (30, 31, 32), CASE valid_tid(32,30,31) WHEN 30 THEN sum(sales) WHEN 31 THEN sum(sales) WHEN 32 THEN sum(sales) END), aggif(valid_tid(32,30,31) IN (30, 31, 32), CASE valid_tid(32,30,31) WHEN 30 THEN sum(`returns`) WHEN 31 THEN sum(`returns`) WHEN 32 THEN sum(`returns`) END), [...]
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=500
@@ -160,15 +160,33 @@ PLAN-ROOT SINK
| | | output: sum(wr_return_amt), sum(wr_net_loss)
| | | group by: wp_web_page_sk
| | | mem-estimate=10.00MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=26 row-size=36B cardinality=3.60K cost=76984674
+| | | tuple-ids=26 row-size=36B cardinality=3.60K cost=1138163
| | | in pipelines: 34(GETNEXT), 29(OPEN)
| | |
| | 33:HASH JOIN [INNER JOIN]
+| | | hash predicates: wr_web_page_sk = wp_web_page_sk
+| | | fk/pk conjuncts: wr_web_page_sk = wp_web_page_sk
+| | | runtime filters: RF016[bloom] <- wp_web_page_sk, RF017[min_max] <- wp_web_page_sk
+| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| | | tuple-ids=23,24,25 row-size=28B cardinality=3.11M cost=1380295
+| | | in pipelines: 29(GETNEXT), 31(OPEN)
+| | |
+| | |--31:SCAN HDFS [tpcds_partitioned_parquet_snap.web_page]
+| | | HDFS partitions=1/1 files=1 size=5.45KB
+| | | stored statistics:
+| | | table: rows=3.60K size=72.76KB
+| | | columns: all
+| | | extrapolated-rows=disabled max-scan-range-rows=3.60K
+| | | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=0
+| | | tuple-ids=25 row-size=4B cardinality=3.60K cost=207
+| | | in pipelines: 31(GETNEXT)
+| | |
+| | 32:HASH JOIN [INNER JOIN]
| | | hash predicates: wr_returned_date_sk = d_date_sk
| | | fk/pk conjuncts: wr_returned_date_sk = d_date_sk
-| | | runtime filters: RF016[bloom] <- d_date_sk
+| | | runtime filters: RF018[bloom] <- d_date_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=23,25,24 row-size=28B cardinality=212.35M cost=92950984
+| | | tuple-ids=23,24 row-size=24B cardinality=3.17M cost=1386443
| | | in pipelines: 29(GETNEXT), 30(OPEN)
| | |
| | |--30:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
@@ -181,52 +199,52 @@ PLAN-ROOT SINK
| | | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | | tuple-ids=24 row-size=8B cardinality=7.30K cost=12520
+| | | tuple-ids=24 row-size=8B cardinality=31 cost=12520
| | | in pipelines: 30(GETNEXT)
| | |
-| | 32:HASH JOIN [INNER JOIN]
-| | | hash predicates: wr_web_page_sk = wp_web_page_sk
-| | | fk/pk conjuncts: wr_web_page_sk = wp_web_page_sk
-| | | runtime filters: RF018[bloom] <- wp_web_page_sk, RF019[min_max] <- wp_web_page_sk
-| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=23,25 row-size=20B cardinality=212.35M cost=92947279
-| | | in pipelines: 29(GETNEXT), 31(OPEN)
-| | |
-| | |--31:SCAN HDFS [tpcds_partitioned_parquet_snap.web_page]
-| | | HDFS partitions=1/1 files=1 size=5.45KB
-| | | stored statistics:
-| | | table: rows=3.60K size=72.76KB
-| | | columns: all
-| | | extrapolated-rows=disabled max-scan-range-rows=3.60K
-| | | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=0
-| | | tuple-ids=25 row-size=4B cardinality=3.60K cost=207
-| | | in pipelines: 31(GETNEXT)
-| | |
| | 29:SCAN HDFS [tpcds_partitioned_parquet_snap.web_returns]
| | HDFS partitions=2114/2114 files=2114 size=20.70MB
-| | runtime filters: RF019[min_max] -> wr_web_page_sk, RF018[bloom] -> wr_web_page_sk, RF016[bloom] -> wr_returned_date_sk
+| | runtime filters: RF017[min_max] -> wr_web_page_sk, RF018[bloom] -> wr_returned_date_sk, RF016[bloom] -> wr_web_page_sk
| | stored statistics:
| | table: rows=216.00M size=16.74GB
| | partitions: 2114/2114 rows=216.00M
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=9.41M
+| | extrapolated-rows=disabled max-scan-range-rows=9.41M est-scan-range=32(filtered from 2114)
| | mem-estimate=16.00MB mem-reservation=256.00KB thread-reservation=0
-| | tuple-ids=23 row-size=16B cardinality=212.35M(filtered from 216.00M) cost=49767018
+| | tuple-ids=23 row-size=16B cardinality=3.17M(filtered from 216.00M) cost=729790
| | in pipelines: 29(GETNEXT)
| |
| 28:AGGREGATE [FINALIZE]
| | output: sum(ws_ext_sales_price), sum(ws_net_profit)
| | group by: wp_web_page_sk
| | mem-estimate=10.00MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=21 row-size=36B cardinality=3.60K cost=769741280
+| | tuple-ids=21 row-size=36B cardinality=3.60K cost=13091452
| | in pipelines: 28(GETNEXT), 23(OPEN)
| |
| 27:HASH JOIN [INNER JOIN]
+| | hash predicates: ws_web_page_sk = wp_web_page_sk
+| | fk/pk conjuncts: ws_web_page_sk = wp_web_page_sk
+| | runtime filters: RF012[bloom] <- wp_web_page_sk, RF013[min_max] <- wp_web_page_sk
+| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=18,19,20 row-size=28B cardinality=36.09M cost=15958939
+| | in pipelines: 23(GETNEXT), 25(OPEN)
+| |
+| |--25:SCAN HDFS [tpcds_partitioned_parquet_snap.web_page]
+| | HDFS partitions=1/1 files=1 size=5.45KB
+| | stored statistics:
+| | table: rows=3.60K size=72.76KB
+| | columns: all
+| | extrapolated-rows=disabled max-scan-range-rows=3.60K
+| | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=0
+| | tuple-ids=20 row-size=4B cardinality=3.60K cost=207
+| | in pipelines: 25(GETNEXT)
+| |
+| 26:HASH JOIN [INNER JOIN]
| | hash predicates: ws_sold_date_sk = d_date_sk
| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| | runtime filters: RF012[bloom] <- d_date_sk
+| | runtime filters: RF014[bloom] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=18,20,19 row-size=28B cardinality=2.12G cost=929418754
+| | tuple-ids=18,19 row-size=24B cardinality=36.71M cost=16067990
| | in pipelines: 23(GETNEXT), 24(OPEN)
| |
| |--24:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
@@ -239,37 +257,19 @@ PLAN-ROOT SINK
| | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=19 row-size=8B cardinality=7.30K cost=12520
+| | tuple-ids=19 row-size=8B cardinality=31 cost=12520
| | in pipelines: 24(GETNEXT)
| |
-| 26:HASH JOIN [INNER JOIN]
-| | hash predicates: ws_web_page_sk = wp_web_page_sk
-| | fk/pk conjuncts: ws_web_page_sk = wp_web_page_sk
-| | runtime filters: RF014[bloom] <- wp_web_page_sk, RF015[min_max] <- wp_web_page_sk
-| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=18,20 row-size=20B cardinality=2.12G cost=929415049
-| | in pipelines: 23(GETNEXT), 25(OPEN)
-| |
-| |--25:SCAN HDFS [tpcds_partitioned_parquet_snap.web_page]
-| | HDFS partitions=1/1 files=1 size=5.45KB
-| | stored statistics:
-| | table: rows=3.60K size=72.76KB
-| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=3.60K
-| | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=0
-| | tuple-ids=20 row-size=4B cardinality=3.60K cost=207
-| | in pipelines: 25(GETNEXT)
-| |
| 23:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales]
| HDFS partitions=1824/1824 files=1824 size=84.18MB
-| runtime filters: RF015[min_max] -> ws_web_page_sk, RF014[bloom] -> ws_web_page_sk, RF012[bloom] -> ws_sold_date_sk
+| runtime filters: RF013[min_max] -> ws_web_page_sk, RF014[bloom] -> ws_sold_date_sk, RF012[bloom] -> ws_web_page_sk
| stored statistics:
| table: rows=2.16G size=145.75GB
| partitions: 1824/1824 rows=2.16G
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=3.37M
+| extrapolated-rows=disabled max-scan-range-rows=3.37M est-scan-range=32(filtered from 1824)
| mem-estimate=16.00MB mem-reservation=128.00KB thread-reservation=0
-| tuple-ids=18 row-size=16B cardinality=2.12G(filtered from 2.16G) cost=497656619
+| tuple-ids=18 row-size=16B cardinality=36.71M(filtered from 2.16G) cost=8457979
| in pipelines: 23(GETNEXT)
|
|--22:NESTED LOOP JOIN [CROSS JOIN]
@@ -281,7 +281,7 @@ PLAN-ROOT SINK
| | | output: sum(cr_return_amount), sum(cr_net_loss)
| | | group by: cr_call_center_sk
| | | mem-estimate=10.00MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=16 row-size=36B cardinality=48 cost=156606284
+| | | tuple-ids=16 row-size=36B cardinality=48 cost=2356819
| | | in pipelines: 21(GETNEXT), 18(OPEN)
| | |
| | 20:HASH JOIN [INNER JOIN]
@@ -289,7 +289,7 @@ PLAN-ROOT SINK
| | | fk/pk conjuncts: cr_returned_date_sk = d_date_sk
| | | runtime filters: RF010[bloom] <- d_date_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=14,15 row-size=24B cardinality=432.02M cost=189101141
+| | | tuple-ids=14,15 row-size=24B cardinality=6.50M cost=2845618
| | | in pipelines: 18(GETNEXT), 19(OPEN)
| | |
| | |--19:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
@@ -302,7 +302,7 @@ PLAN-ROOT SINK
| | | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | | tuple-ids=15 row-size=8B cardinality=7.30K cost=12520
+| | | tuple-ids=15 row-size=8B cardinality=31 cost=12520
| | | in pipelines: 19(GETNEXT)
| | |
| | 18:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_returns]
@@ -312,16 +312,16 @@ PLAN-ROOT SINK
| | table: rows=432.02M size=32.77GB
| | partitions: 2060/2060 rows=432.02M
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=416.82K
+| | extrapolated-rows=disabled max-scan-range-rows=416.82K est-scan-range=32(filtered from 2060)
| | mem-estimate=16.00MB mem-reservation=32.00KB thread-reservation=0
-| | tuple-ids=14 row-size=16B cardinality=432.02M cost=99536714
+| | tuple-ids=14 row-size=16B cardinality=6.50M(filtered from 432.02M) cost=1497882
| | in pipelines: 18(GETNEXT)
| |
| 17:AGGREGATE [FINALIZE]
| | output: sum(cs_ext_sales_price), sum(cs_net_profit)
| | group by: cs_call_center_sk
| | mem-estimate=10.00MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=12 row-size=36B cardinality=48 cost=1566028392
+| | tuple-ids=12 row-size=36B cardinality=48 cost=26513984
| | in pipelines: 17(GETNEXT), 14(OPEN)
| |
| 16:HASH JOIN [INNER JOIN]
@@ -329,7 +329,7 @@ PLAN-ROOT SINK
| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
| | runtime filters: RF008[bloom] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=10,11 row-size=24B cardinality=4.32G cost=1890905436
+| | tuple-ids=10,11 row-size=24B cardinality=73.14M cost=32014144
| | in pipelines: 14(GETNEXT), 15(OPEN)
| |
| |--15:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
@@ -342,7 +342,7 @@ PLAN-ROOT SINK
| | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=11 row-size=8B cardinality=7.30K cost=12520
+| | tuple-ids=11 row-size=8B cardinality=31 cost=12520
| | in pipelines: 15(GETNEXT)
| |
| 14:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales]
@@ -352,9 +352,9 @@ PLAN-ROOT SINK
| table: rows=4.32G size=280.96GB
| partitions: 1831/1831 rows=4.32G
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=21.52M
+| extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=31(filtered from 1831)
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=10 row-size=16B cardinality=4.32G cost=995345966
+| tuple-ids=10 row-size=16B cardinality=73.14M(filtered from 4.32G) cost=16851843
| in pipelines: 14(GETNEXT)
|
13:HASH JOIN [LEFT OUTER JOIN]
@@ -368,7 +368,7 @@ PLAN-ROOT SINK
| | output: sum(sr_return_amt), sum(sr_net_loss)
| | group by: s_store_sk
| | mem-estimate=10.00MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=8 row-size=36B cardinality=1.32K cost=313199317
+| | tuple-ids=8 row-size=36B cardinality=1.32K cost=4848285
| | in pipelines: 12(GETNEXT), 07(OPEN)
| |
| 11:HASH JOIN [INNER JOIN]
@@ -376,7 +376,7 @@ PLAN-ROOT SINK
| | fk/pk conjuncts: sr_store_sk = s_store_sk
| | runtime filters: RF004[bloom] <- s_store_sk, RF005[min_max] <- s_store_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=5,6,7 row-size=28B cardinality=863.99M cost=378169174
+| | tuple-ids=5,6,7 row-size=28B cardinality=13.37M cost=5851251
| | in pipelines: 07(GETNEXT), 09(OPEN)
| |
| |--09:SCAN HDFS [tpcds_partitioned_parquet_snap.store]
@@ -394,7 +394,7 @@ PLAN-ROOT SINK
| | fk/pk conjuncts: sr_returned_date_sk = d_date_sk
| | runtime filters: RF006[bloom] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=5,6 row-size=24B cardinality=863.99M cost=378175129
+| | tuple-ids=5,6 row-size=24B cardinality=13.37M cost=5849932
| | in pipelines: 07(GETNEXT), 08(OPEN)
| |
| |--08:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
@@ -407,26 +407,26 @@ PLAN-ROOT SINK
| | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=6 row-size=8B cardinality=7.30K cost=12520
+| | tuple-ids=6 row-size=8B cardinality=31 cost=12520
| | in pipelines: 08(GETNEXT)
| |
| 07:SCAN HDFS [tpcds_partitioned_parquet_snap.store_returns]
| HDFS partitions=2004/2004 files=2004 size=33.63MB
-| runtime filters: RF005[min_max] -> sr_store_sk, RF004[bloom] -> sr_store_sk, RF006[bloom] -> sr_returned_date_sk
+| runtime filters: RF005[min_max] -> sr_store_sk, RF006[bloom] -> sr_returned_date_sk, RF004[bloom] -> sr_store_sk
| stored statistics:
| table: rows=863.99M size=48.14GB
| partitions: 2004/2004 rows=863.99M
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=30.09M
+| extrapolated-rows=disabled max-scan-range-rows=30.09M est-scan-range=32(filtered from 2004)
| mem-estimate=16.00MB mem-reservation=512.00KB thread-reservation=0
-| tuple-ids=5 row-size=16B cardinality=863.99M cost=199062980
+| tuple-ids=5 row-size=16B cardinality=13.37M(filtered from 863.99M) cost=3079317
| in pipelines: 07(GETNEXT)
|
06:AGGREGATE [FINALIZE]
| output: sum(ss_ext_sales_price), sum(ss_net_profit)
| group by: s_store_sk
| mem-estimate=10.00MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=3 row-size=36B cardinality=1.32K cost=3131979946
+| tuple-ids=3 row-size=36B cardinality=1.32K cost=53233302
| in pipelines: 06(GETNEXT), 01(OPEN)
|
05:HASH JOIN [INNER JOIN]
@@ -434,7 +434,7 @@ PLAN-ROOT SINK
| fk/pk conjuncts: ss_store_sk = s_store_sk
| runtime filters: RF000[bloom] <- s_store_sk, RF001[min_max] <- s_store_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1,2 row-size=28B cardinality=8.64G cost=3781700983
+| tuple-ids=0,1,2 row-size=28B cardinality=146.84M cost=64273657
| in pipelines: 01(GETNEXT), 03(OPEN)
|
|--03:SCAN HDFS [tpcds_partitioned_parquet_snap.store]
@@ -452,7 +452,7 @@ PLAN-ROOT SINK
| fk/pk conjuncts: ss_sold_date_sk = d_date_sk
| runtime filters: RF002[bloom] <- d_date_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1 row-size=24B cardinality=8.64G cost=3781706938
+| tuple-ids=0,1 row-size=24B cardinality=146.84M cost=64272338
| in pipelines: 01(GETNEXT), 02(OPEN)
|
|--02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
@@ -465,26 +465,26 @@ PLAN-ROOT SINK
| parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=1 row-size=8B cardinality=7.30K cost=12520
+| tuple-ids=1 row-size=8B cardinality=31 cost=12520
| in pipelines: 02(GETNEXT)
|
01:SCAN HDFS [tpcds_partitioned_parquet_snap.store_sales]
HDFS partitions=1824/1824 files=1824 size=199.33MB
- runtime filters: RF001[min_max] -> ss_store_sk, RF000[bloom] -> ss_store_sk, RF002[bloom] -> ss_sold_date_sk
+ runtime filters: RF001[min_max] -> ss_store_sk, RF002[bloom] -> ss_sold_date_sk, RF000[bloom] -> ss_store_sk
stored statistics:
table: rows=8.64G size=389.90GB
partitions: 1824/1824 rows=8.64G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=390.22M
+ extrapolated-rows=disabled max-scan-range-rows=390.22M est-scan-range=32(filtered from 1824)
mem-estimate=16.00MB mem-reservation=8.00MB thread-reservation=0
- tuple-ids=0 row-size=16B cardinality=8.64G cost=1990641068
+ tuple-ids=0 row-size=16B cardinality=146.84M(filtered from 8.64G) cost=33832167
in pipelines: 01(GETNEXT)
---- DISTRIBUTEDPLAN
-Max Per-Host Resource Reservation: Memory=646.58MB Threads=117
-Per-Host Resource Estimates: Memory=3.76GB
+Max Per-Host Resource Reservation: Memory=155.83MB Threads=41
+Per-Host Resource Estimates: Memory=723MB
F24:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=4.07MB mem-reservation=4.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[538] cpu-comparison-result=650 [max(1 (self) vs 650 (sum children))]
+| max-parallelism=1 segment-costs=[538] cpu-comparison-result=134 [max(1 (self) vs 134 (sum children))]
PLAN-ROOT SINK
| output exprs: CASE valid_tid(32,30,31) WHEN 30 THEN channel WHEN 31 THEN channel WHEN 32 THEN NULL END, CASE valid_tid(32,30,31) WHEN 30 THEN id WHEN 31 THEN NULL WHEN 32 THEN NULL END, aggif(valid_tid(32,30,31) IN (30, 31, 32), CASE valid_tid(32,30,31) WHEN 30 THEN sum(sales) WHEN 31 THEN sum(sales) WHEN 32 THEN sum(sales) END), aggif(valid_tid(32,30,31) IN (30, 31, 32), CASE valid_tid(32,30,31) WHEN 30 THEN sum(`returns`) WHEN 31 THEN sum(`returns`) WHEN 32 THEN sum(`returns`) END), [...]
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=500
@@ -497,8 +497,8 @@ PLAN-ROOT SINK
| in pipelines: 38(GETNEXT)
|
F23:PLAN FRAGMENT [HASH(CASE valid_tid(32,30,31) WHEN 30 THEN murmur_hash(channel) WHEN 31 THEN murmur_hash(channel) WHEN 32 THEN murmur_hash(NULL) END,CASE valid_tid(32,30,31) WHEN 30 THEN murmur_hash(id) WHEN 31 THEN murmur_hash(NULL) WHEN 32 THEN murmur_hash(NULL) END)] hosts=10 instances=10 (adjusted from 120)
-Per-Instance Resources: mem-estimate=54.04MB mem-reservation=7.75MB thread-reservation=1
-max-parallelism=10 segment-costs=[51386, 43669, 52232, 199] cpu-comparison-result=650 [max(10 (self) vs 650 (sum children))]
+Per-Instance Resources: mem-estimate=40.00MB mem-reservation=7.75MB thread-reservation=1
+max-parallelism=10 segment-costs=[51386, 43669, 52232, 199] cpu-comparison-result=134 [max(10 (self) vs 134 (sum children))]
38:TOP-N [LIMIT=100]
| order by: CASE valid_tid(32,30,31) WHEN 30 THEN channel WHEN 31 THEN channel WHEN 32 THEN NULL END ASC, CASE valid_tid(32,30,31) WHEN 30 THEN id WHEN 31 THEN NULL WHEN 32 THEN NULL END ASC
| mem-estimate=6.25KB mem-reservation=0B thread-reservation=0
@@ -527,13 +527,13 @@ max-parallelism=10 segment-costs=[51386, 43669, 52232, 199] cpu-comparison-resul
| in pipelines: 63(GETNEXT), 42(OPEN), 49(OPEN), 57(OPEN)
|
62:EXCHANGE [HASH(CASE valid_tid(32,30,31) WHEN 30 THEN murmur_hash(channel) WHEN 31 THEN murmur_hash(channel) WHEN 32 THEN murmur_hash(NULL) END,CASE valid_tid(32,30,31) WHEN 30 THEN murmur_hash(id) WHEN 31 THEN murmur_hash(NULL) WHEN 32 THEN murmur_hash(NULL) END)]
-| mem-estimate=24.04MB mem-reservation=0B thread-reservation=0
+| mem-estimate=6.11MB mem-reservation=0B thread-reservation=0
| tuple-ids=30N,31N,32N row-size=192B cardinality=7.23K cost=7717
| in pipelines: 42(GETNEXT), 49(GETNEXT), 57(GETNEXT)
|
-F22:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=52.67MB mem-reservation=7.94MB thread-reservation=1
-max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparison-result=650 [max(120 (self) vs 650 (sum children))]
+F22:PLAN FRAGMENT [RANDOM] hosts=10 instances=30 (adjusted from 120)
+Per-Instance Resources: mem-estimate=49.15MB mem-reservation=7.94MB thread-reservation=1
+max-parallelism=30 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparison-result=134 [max(30 (self) vs 134 (sum children))]
36:AGGREGATE [STREAMING]
| Class 0
| output: sum(sales), sum(returns), sum(profit)
@@ -561,9 +561,9 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | tuple-ids=21,26N row-size=72B cardinality=3.60K cost=1575
| | in pipelines: 57(GETNEXT), 61(OPEN)
| |
-| |--F30:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+| |--F30:PLAN FRAGMENT [RANDOM] hosts=10 instances=30 (adjusted from 120)
| | | Per-Instance Resources: mem-estimate=11.94MB mem-reservation=3.88MB thread-reservation=1
-| | | max-parallelism=120 segment-costs=[11374, 3600] cpu-comparison-result=120 [max(120 (self) vs 40 (sum children))]
+| | | max-parallelism=30 segment-costs=[11374, 3600] cpu-comparison-result=30 [max(30 (self) vs 22 (sum children))]
| | JOIN BUILD
| | | join-table-id=05 plan-id=06 cohort-id=01
| | | build expressions: wp_web_page_sk
@@ -577,104 +577,104 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | | in pipelines: 61(GETNEXT), 29(OPEN)
| | |
| | 60:EXCHANGE [HASH(wp_web_page_sk)]
-| | | mem-estimate=1.57MB mem-reservation=0B thread-reservation=0
+| | | mem-estimate=412.66KB mem-reservation=0B thread-reservation=0
| | | tuple-ids=26 row-size=36B cardinality=3.60K cost=996
| | | in pipelines: 29(GETNEXT)
| | |
-| | F18:PLAN FRAGMENT [RANDOM] hosts=10 instances=40 (adjusted from 120)
+| | F18:PLAN FRAGMENT [RANDOM] hosts=10 instances=10 (adjusted from 120)
| | Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
-| | Per-Instance Resources: mem-estimate=44.75MB mem-reservation=2.25MB thread-reservation=1
-| | max-parallelism=40 segment-costs=[313755974, 10200] cpu-comparison-result=40 [max(40 (self) vs 22 (sum children))]
+| | Per-Instance Resources: mem-estimate=30.69MB mem-reservation=2.25MB thread-reservation=1
+| | max-parallelism=10 segment-costs=[5747045, 10200] cpu-comparison-result=22 [max(10 (self) vs 22 (sum children))]
| | 34:AGGREGATE [STREAMING]
| | | output: sum(wr_return_amt), sum(wr_net_loss)
| | | group by: wp_web_page_sk
| | | mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=26 row-size=36B cardinality=3.60K cost=78101598
+| | | tuple-ids=26 row-size=36B cardinality=3.60K cost=2254148
| | | in pipelines: 29(GETNEXT)
| | |
| | 33:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash-table-id=06
-| | | hash predicates: wr_returned_date_sk = d_date_sk
-| | | fk/pk conjuncts: wr_returned_date_sk = d_date_sk
+| | | hash predicates: wr_web_page_sk = wp_web_page_sk
+| | | fk/pk conjuncts: wr_web_page_sk = wp_web_page_sk
| | | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=23,25,24 row-size=28B cardinality=212.35M cost=92943679
-| | | in pipelines: 29(GETNEXT), 30(OPEN)
+| | | tuple-ids=23,24,25 row-size=28B cardinality=3.11M cost=1376695
+| | | in pipelines: 29(GETNEXT), 31(OPEN)
| | |
| | |--F31:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | | Per-Instance Resources: mem-estimate=8.82MB mem-reservation=8.75MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | | max-parallelism=10 segment-costs=[17005]
+| | | | Per-Instance Resources: mem-estimate=2.96MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | | max-parallelism=10 segment-costs=[8380]
| | | JOIN BUILD
| | | | join-table-id=06 plan-id=07 cohort-id=04
-| | | | build expressions: d_date_sk
-| | | | runtime filters: RF016[bloom] <- d_date_sk
-| | | | mem-estimate=7.75MB mem-reservation=7.75MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | | | build expressions: wp_web_page_sk
+| | | | runtime filters: RF016[bloom] <- wp_web_page_sk, RF017[min_max] <- wp_web_page_sk
+| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=3600
| | | |
| | | 59:EXCHANGE [BROADCAST]
-| | | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | | tuple-ids=24 row-size=8B cardinality=7.30K cost=9700
-| | | | in pipelines: 30(GETNEXT)
+| | | | mem-estimate=22.06KB mem-reservation=0B thread-reservation=0
+| | | | tuple-ids=25 row-size=4B cardinality=3.60K cost=4780
+| | | | in pipelines: 31(GETNEXT)
| | | |
| | | F20:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | | max-parallelism=1 segment-costs=[12835]
-| | | 30:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| | | HDFS partitions=1/1 files=1 size=2.17MB
-| | | predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
+| | | Per-Instance Resources: mem-estimate=16.03MB mem-reservation=8.00KB thread-reservation=1
+| | | max-parallelism=1 segment-costs=[323]
+| | | 31:SCAN HDFS [tpcds_partitioned_parquet_snap.web_page, RANDOM]
+| | | HDFS partitions=1/1 files=1 size=5.45KB
| | | stored statistics:
-| | | table: rows=73.05K size=2.17MB
+| | | table: rows=3.60K size=72.76KB
| | | columns: all
-| | | extrapolated-rows=disabled max-scan-range-rows=73.05K
-| | | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
-| | | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
-| | | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | | tuple-ids=24 row-size=8B cardinality=7.30K cost=12520
-| | | in pipelines: 30(GETNEXT)
+| | | extrapolated-rows=disabled max-scan-range-rows=3.60K
+| | | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=0
+| | | tuple-ids=25 row-size=4B cardinality=3.60K cost=207
+| | | in pipelines: 31(GETNEXT)
| | |
| | 32:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash-table-id=07
-| | | hash predicates: wr_web_page_sk = wp_web_page_sk
-| | | fk/pk conjuncts: wr_web_page_sk = wp_web_page_sk
+| | | hash predicates: wr_returned_date_sk = d_date_sk
+| | | fk/pk conjuncts: wr_returned_date_sk = d_date_sk
| | | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=23,25 row-size=20B cardinality=212.35M cost=92943679
-| | | in pipelines: 29(GETNEXT), 31(OPEN)
+| | | tuple-ids=23,24 row-size=24B cardinality=3.17M cost=1386412
+| | | in pipelines: 29(GETNEXT), 30(OPEN)
| | |
| | |--F32:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | | Per-Instance Resources: mem-estimate=8.77MB mem-reservation=8.75MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | | max-parallelism=10 segment-costs=[8380]
+| | | | Per-Instance Resources: mem-estimate=2.95MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | | max-parallelism=10 segment-costs=[71]
| | | JOIN BUILD
| | | | join-table-id=07 plan-id=08 cohort-id=04
-| | | | build expressions: wp_web_page_sk
-| | | | runtime filters: RF018[bloom] <- wp_web_page_sk, RF019[min_max] <- wp_web_page_sk
-| | | | mem-estimate=7.75MB mem-reservation=7.75MB spill-buffer=64.00KB thread-reservation=0 cost=3600
+| | | | build expressions: d_date_sk
+| | | | runtime filters: RF018[bloom] <- d_date_sk
+| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=31
| | | |
| | | 58:EXCHANGE [BROADCAST]
-| | | | mem-estimate=22.06KB mem-reservation=0B thread-reservation=0
-| | | | tuple-ids=25 row-size=4B cardinality=3.60K cost=4780
-| | | | in pipelines: 31(GETNEXT)
+| | | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | | tuple-ids=24 row-size=8B cardinality=31 cost=40
+| | | | in pipelines: 30(GETNEXT)
| | | |
| | | F19:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | | Per-Instance Resources: mem-estimate=16.03MB mem-reservation=8.00KB thread-reservation=1
-| | | max-parallelism=1 segment-costs=[323]
-| | | 31:SCAN HDFS [tpcds_partitioned_parquet_snap.web_page, RANDOM]
-| | | HDFS partitions=1/1 files=1 size=5.45KB
+| | | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| | | max-parallelism=1 segment-costs=[12521]
+| | | 30:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| | | HDFS partitions=1/1 files=1 size=2.17MB
+| | | predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | | stored statistics:
-| | | table: rows=3.60K size=72.76KB
+| | | table: rows=73.05K size=2.17MB
| | | columns: all
-| | | extrapolated-rows=disabled max-scan-range-rows=3.60K
-| | | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=0
-| | | tuple-ids=25 row-size=4B cardinality=3.60K cost=207
-| | | in pipelines: 31(GETNEXT)
+| | | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
+| | | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
+| | | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | | tuple-ids=24 row-size=8B cardinality=31 cost=12520
+| | | in pipelines: 30(GETNEXT)
| | |
| | 29:SCAN HDFS [tpcds_partitioned_parquet_snap.web_returns, RANDOM]
| | HDFS partitions=2114/2114 files=2114 size=20.70MB
-| | runtime filters: RF019[min_max] -> wr_web_page_sk, RF018[bloom] -> wr_web_page_sk, RF016[bloom] -> wr_returned_date_sk
+| | runtime filters: RF017[min_max] -> wr_web_page_sk, RF018[bloom] -> wr_returned_date_sk, RF016[bloom] -> wr_web_page_sk
| | stored statistics:
| | table: rows=216.00M size=16.74GB
| | partitions: 2114/2114 rows=216.00M
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=9.41M
+| | extrapolated-rows=disabled max-scan-range-rows=9.41M est-scan-range=32(filtered from 2114)
| | mem-estimate=16.00MB mem-reservation=256.00KB thread-reservation=0
-| | tuple-ids=23 row-size=16B cardinality=212.35M(filtered from 216.00M) cost=49767018
+| | tuple-ids=23 row-size=16B cardinality=3.17M(filtered from 216.00M) cost=729790
| | in pipelines: 29(GETNEXT)
| |
| 57:AGGREGATE [FINALIZE]
@@ -685,104 +685,104 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | in pipelines: 57(GETNEXT), 23(OPEN)
| |
| 56:EXCHANGE [HASH(wp_web_page_sk)]
-| | mem-estimate=4.70MB mem-reservation=0B thread-reservation=0
+| | mem-estimate=412.66KB mem-reservation=0B thread-reservation=0
| | tuple-ids=21 row-size=36B cardinality=3.60K cost=996
| | in pipelines: 23(GETNEXT)
| |
-| F14:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+| F14:PLAN FRAGMENT [RANDOM] hosts=10 instances=10 (adjusted from 120)
| Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
-| Per-Instance Resources: mem-estimate=44.75MB mem-reservation=2.12MB thread-reservation=1
-| max-parallelism=320 segment-costs=[3127337722, 10200] cpu-comparison-result=120 [max(120 (self) vs 22 (sum children))]
+| Per-Instance Resources: mem-estimate=30.69MB mem-reservation=2.12MB thread-reservation=1
+| max-parallelism=10 segment-costs=[54689653, 10200] cpu-comparison-result=22 [max(10 (self) vs 22 (sum children))]
| 28:AGGREGATE [STREAMING]
| | output: sum(ws_ext_sales_price), sum(ws_net_profit)
| | group by: wp_web_page_sk
| | mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=21 row-size=36B cardinality=3.60K cost=770858205
+| | tuple-ids=21 row-size=36B cardinality=3.60K cost=14208376
| | in pipelines: 23(GETNEXT)
| |
| 27:HASH JOIN [INNER JOIN, BROADCAST]
| | hash-table-id=11
-| | hash predicates: ws_sold_date_sk = d_date_sk
-| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
+| | hash predicates: ws_web_page_sk = wp_web_page_sk
+| | fk/pk conjuncts: ws_web_page_sk = wp_web_page_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=18,20,19 row-size=28B cardinality=2.12G cost=929411449
-| | in pipelines: 23(GETNEXT), 24(OPEN)
+| | tuple-ids=18,19,20 row-size=28B cardinality=36.09M cost=15955339
+| | in pipelines: 23(GETNEXT), 25(OPEN)
| |
| |--F36:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[17005]
+| | | Per-Instance Resources: mem-estimate=2.96MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[8380]
| | JOIN BUILD
| | | join-table-id=11 plan-id=12 cohort-id=01
-| | | build expressions: d_date_sk
-| | | runtime filters: RF012[bloom] <- d_date_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | | build expressions: wp_web_page_sk
+| | | runtime filters: RF012[bloom] <- wp_web_page_sk, RF013[min_max] <- wp_web_page_sk
+| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=3600
| | |
| | 55:EXCHANGE [BROADCAST]
-| | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=19 row-size=8B cardinality=7.30K cost=9700
-| | | in pipelines: 24(GETNEXT)
+| | | mem-estimate=22.06KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=20 row-size=4B cardinality=3.60K cost=4780
+| | | in pipelines: 25(GETNEXT)
| | |
| | F16:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[12835]
-| | 24:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| | HDFS partitions=1/1 files=1 size=2.17MB
-| | predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
+| | Per-Instance Resources: mem-estimate=16.03MB mem-reservation=8.00KB thread-reservation=1
+| | max-parallelism=1 segment-costs=[323]
+| | 25:SCAN HDFS [tpcds_partitioned_parquet_snap.web_page, RANDOM]
+| | HDFS partitions=1/1 files=1 size=5.45KB
| | stored statistics:
-| | table: rows=73.05K size=2.17MB
+| | table: rows=3.60K size=72.76KB
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=73.05K
-| | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
-| | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
-| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=19 row-size=8B cardinality=7.30K cost=12520
-| | in pipelines: 24(GETNEXT)
+| | extrapolated-rows=disabled max-scan-range-rows=3.60K
+| | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=0
+| | tuple-ids=20 row-size=4B cardinality=3.60K cost=207
+| | in pipelines: 25(GETNEXT)
| |
| 26:HASH JOIN [INNER JOIN, BROADCAST]
| | hash-table-id=12
-| | hash predicates: ws_web_page_sk = wp_web_page_sk
-| | fk/pk conjuncts: ws_web_page_sk = wp_web_page_sk
+| | hash predicates: ws_sold_date_sk = d_date_sk
+| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=18,20 row-size=20B cardinality=2.12G cost=929411449
-| | in pipelines: 23(GETNEXT), 25(OPEN)
+| | tuple-ids=18,19 row-size=24B cardinality=36.71M cost=16067959
+| | in pipelines: 23(GETNEXT), 24(OPEN)
| |
| |--F37:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[8380]
+| | | Per-Instance Resources: mem-estimate=2.95MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[71]
| | JOIN BUILD
| | | join-table-id=12 plan-id=13 cohort-id=01
-| | | build expressions: wp_web_page_sk
-| | | runtime filters: RF014[bloom] <- wp_web_page_sk, RF015[min_max] <- wp_web_page_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=3600
+| | | build expressions: d_date_sk
+| | | runtime filters: RF014[bloom] <- d_date_sk
+| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=31
| | |
| | 54:EXCHANGE [BROADCAST]
-| | | mem-estimate=22.06KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=20 row-size=4B cardinality=3.60K cost=4780
-| | | in pipelines: 25(GETNEXT)
+| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=19 row-size=8B cardinality=31 cost=40
+| | | in pipelines: 24(GETNEXT)
| | |
| | F15:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | Per-Instance Resources: mem-estimate=16.03MB mem-reservation=8.00KB thread-reservation=1
-| | max-parallelism=1 segment-costs=[323]
-| | 25:SCAN HDFS [tpcds_partitioned_parquet_snap.web_page, RANDOM]
-| | HDFS partitions=1/1 files=1 size=5.45KB
+| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| | max-parallelism=1 segment-costs=[12521]
+| | 24:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| | HDFS partitions=1/1 files=1 size=2.17MB
+| | predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | stored statistics:
-| | table: rows=3.60K size=72.76KB
+| | table: rows=73.05K size=2.17MB
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=3.60K
-| | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=0
-| | tuple-ids=20 row-size=4B cardinality=3.60K cost=207
-| | in pipelines: 25(GETNEXT)
+| | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
+| | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
+| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | tuple-ids=19 row-size=8B cardinality=31 cost=12520
+| | in pipelines: 24(GETNEXT)
| |
| 23:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales, RANDOM]
| HDFS partitions=1824/1824 files=1824 size=84.18MB
-| runtime filters: RF015[min_max] -> ws_web_page_sk, RF014[bloom] -> ws_web_page_sk, RF012[bloom] -> ws_sold_date_sk
+| runtime filters: RF013[min_max] -> ws_web_page_sk, RF014[bloom] -> ws_sold_date_sk, RF012[bloom] -> ws_web_page_sk
| stored statistics:
| table: rows=2.16G size=145.75GB
| partitions: 1824/1824 rows=2.16G
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=3.37M
+| extrapolated-rows=disabled max-scan-range-rows=3.37M est-scan-range=32(filtered from 1824)
| mem-estimate=16.00MB mem-reservation=128.00KB thread-reservation=0
-| tuple-ids=18 row-size=16B cardinality=2.12G(filtered from 2.16G) cost=497656619
+| tuple-ids=18 row-size=16B cardinality=36.71M(filtered from 2.16G) cost=8457979
| in pipelines: 23(GETNEXT)
|
|--22:NESTED LOOP JOIN [CROSS JOIN, BROADCAST]
@@ -793,7 +793,7 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| |
| |--F28:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
| | | Per-Instance Resources: mem-estimate=22.12KB mem-reservation=0B thread-reservation=1
-| | | max-parallelism=10 segment-costs=[60] cpu-comparison-result=50 [max(10 (self) vs 50 (sum children))]
+| | | max-parallelism=10 segment-costs=[60] cpu-comparison-result=11 [max(10 (self) vs 11 (sum children))]
| | JOIN BUILD
| | | join-table-id=03 plan-id=04 cohort-id=01
| | | mem-estimate=1.69KB mem-reservation=0B thread-reservation=0 cost=0
@@ -805,7 +805,7 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | |
| | F13:PLAN FRAGMENT [HASH(cr_call_center_sk)] hosts=10 instances=10 (adjusted from 120)
| | Per-Instance Resources: mem-estimate=10.16MB mem-reservation=1.94MB thread-reservation=1
-| | max-parallelism=10 segment-costs=[155, 5] cpu-comparison-result=50 [max(10 (self) vs 50 (sum children))]
+| | max-parallelism=10 segment-costs=[155, 5] cpu-comparison-result=11 [max(10 (self) vs 11 (sum children))]
| | 52:AGGREGATE [FINALIZE]
| | | output: sum:merge(cr_return_amount), sum:merge(cr_net_loss)
| | | group by: cr_call_center_sk
@@ -814,19 +814,19 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | | in pipelines: 52(GETNEXT), 18(OPEN)
| | |
| | 51:EXCHANGE [HASH(cr_call_center_sk)]
-| | | mem-estimate=93.92KB mem-reservation=0B thread-reservation=0
+| | | mem-estimate=18.92KB mem-reservation=0B thread-reservation=0
| | | tuple-ids=16 row-size=36B cardinality=48 cost=13
| | | in pipelines: 18(GETNEXT)
| | |
-| | F11:PLAN FRAGMENT [RANDOM] hosts=10 instances=50 (adjusted from 120)
+| | F11:PLAN FRAGMENT [RANDOM] hosts=10 instances=10 (adjusted from 120)
| | Per-Host Shared Resources: mem-estimate=1.00MB mem-reservation=1.00MB thread-reservation=0 runtime-filters-memory=1.00MB
| | Per-Instance Resources: mem-estimate=27.56MB mem-reservation=2.03MB thread-reservation=1
-| | max-parallelism=50 segment-costs=[445251726, 136] cpu-comparison-result=50 [max(50 (self) vs 11 (sum children))]
+| | max-parallelism=10 segment-costs=[6715181, 136] cpu-comparison-result=11 [max(10 (self) vs 11 (sum children))]
| | 21:AGGREGATE [STREAMING]
| | | output: sum(cr_return_amount), sum(cr_net_loss)
| | | group by: cr_call_center_sk
| | | mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=16 row-size=36B cardinality=48 cost=156621176
+| | | tuple-ids=16 row-size=36B cardinality=48 cost=2371712
| | | in pipelines: 18(GETNEXT)
| | |
| | 20:HASH JOIN [INNER JOIN, BROADCAST]
@@ -834,26 +834,26 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | | hash predicates: cr_returned_date_sk = d_date_sk
| | | fk/pk conjuncts: cr_returned_date_sk = d_date_sk
| | | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=14,15 row-size=24B cardinality=432.02M cost=189093836
+| | | tuple-ids=14,15 row-size=24B cardinality=6.50M cost=2845587
| | | in pipelines: 18(GETNEXT), 19(OPEN)
| | |
| | |--F29:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | | Per-Instance Resources: mem-estimate=10.75MB mem-reservation=10.69MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | | max-parallelism=10 segment-costs=[17005]
+| | | | Per-Instance Resources: mem-estimate=2.95MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | | max-parallelism=10 segment-costs=[71]
| | | JOIN BUILD
| | | | join-table-id=04 plan-id=05 cohort-id=03
| | | | build expressions: d_date_sk
| | | | runtime filters: RF010[bloom] <- d_date_sk
-| | | | mem-estimate=9.69MB mem-reservation=9.69MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=31
| | | |
| | | 50:EXCHANGE [BROADCAST]
-| | | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | | tuple-ids=15 row-size=8B cardinality=7.30K cost=9700
+| | | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | | tuple-ids=15 row-size=8B cardinality=31 cost=40
| | | | in pipelines: 19(GETNEXT)
| | | |
| | | F12:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| | | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | | max-parallelism=1 segment-costs=[12835]
+| | | max-parallelism=1 segment-costs=[12521]
| | | 19:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| | | HDFS partitions=1/1 files=1 size=2.17MB
| | | predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
@@ -864,7 +864,7 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | | tuple-ids=15 row-size=8B cardinality=7.30K cost=12520
+| | | tuple-ids=15 row-size=8B cardinality=31 cost=12520
| | | in pipelines: 19(GETNEXT)
| | |
| | 18:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_returns, RANDOM]
@@ -874,9 +874,9 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | table: rows=432.02M size=32.77GB
| | partitions: 2060/2060 rows=432.02M
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=416.82K
+| | extrapolated-rows=disabled max-scan-range-rows=416.82K est-scan-range=32(filtered from 2060)
| | mem-estimate=16.00MB mem-reservation=32.00KB thread-reservation=0
-| | tuple-ids=14 row-size=16B cardinality=432.02M cost=99536714
+| | tuple-ids=14 row-size=16B cardinality=6.50M(filtered from 432.02M) cost=1497882
| | in pipelines: 18(GETNEXT)
| |
| 49:AGGREGATE [FINALIZE]
@@ -887,19 +887,19 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | in pipelines: 49(GETNEXT), 14(OPEN)
| |
| 48:EXCHANGE [HASH(cs_call_center_sk)]
-| | mem-estimate=225.17KB mem-reservation=0B thread-reservation=0
+| | mem-estimate=18.92KB mem-reservation=0B thread-reservation=0
| | tuple-ids=12 row-size=36B cardinality=48 cost=13
| | in pipelines: 14(GETNEXT)
| |
-| F08:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+| F08:PLAN FRAGMENT [RANDOM] hosts=10 instances=10 (adjusted from 120)
| Per-Host Shared Resources: mem-estimate=1.00MB mem-reservation=1.00MB thread-reservation=0 runtime-filters-memory=1.00MB
-| Per-Instance Resources: mem-estimate=44.75MB mem-reservation=3.00MB thread-reservation=1
-| max-parallelism=450 segment-costs=[4452287381, 136] cpu-comparison-result=120 [max(120 (self) vs 11 (sum children))]
+| Per-Instance Resources: mem-estimate=30.69MB mem-reservation=3.00MB thread-reservation=1
+| max-parallelism=10 segment-costs=[75394832, 136] cpu-comparison-result=11 [max(10 (self) vs 11 (sum children))]
| 17:AGGREGATE [STREAMING]
| | output: sum(cs_ext_sales_price), sum(cs_net_profit)
| | group by: cs_call_center_sk
| | mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=12 row-size=36B cardinality=48 cost=1566043284
+| | tuple-ids=12 row-size=36B cardinality=48 cost=26528876
| | in pipelines: 14(GETNEXT)
| |
| 16:HASH JOIN [INNER JOIN, BROADCAST]
@@ -907,26 +907,26 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | hash predicates: cs_sold_date_sk = d_date_sk
| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=10,11 row-size=24B cardinality=4.32G cost=1890898131
+| | tuple-ids=10,11 row-size=24B cardinality=73.14M cost=32014113
| | in pipelines: 14(GETNEXT), 15(OPEN)
| |
| |--F35:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[17005]
+| | | Per-Instance Resources: mem-estimate=2.95MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[71]
| | JOIN BUILD
| | | join-table-id=10 plan-id=11 cohort-id=01
| | | build expressions: d_date_sk
| | | runtime filters: RF008[bloom] <- d_date_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=31
| | |
| | 47:EXCHANGE [BROADCAST]
-| | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=11 row-size=8B cardinality=7.30K cost=9700
+| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=11 row-size=8B cardinality=31 cost=40
| | | in pipelines: 15(GETNEXT)
| | |
| | F09:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[12835]
+| | max-parallelism=1 segment-costs=[12521]
| | 15:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| | HDFS partitions=1/1 files=1 size=2.17MB
| | predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
@@ -937,7 +937,7 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=11 row-size=8B cardinality=7.30K cost=12520
+| | tuple-ids=11 row-size=8B cardinality=31 cost=12520
| | in pipelines: 15(GETNEXT)
| |
| 14:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales, RANDOM]
@@ -947,9 +947,9 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| table: rows=4.32G size=280.96GB
| partitions: 1831/1831 rows=4.32G
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=21.52M
+| extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=31(filtered from 1831)
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=10 row-size=16B cardinality=4.32G cost=995345966
+| tuple-ids=10 row-size=16B cardinality=73.14M(filtered from 4.32G) cost=16851843
| in pipelines: 14(GETNEXT)
|
13:HASH JOIN [LEFT OUTER JOIN, PARTITIONED]
@@ -960,9 +960,9 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| tuple-ids=3,8N row-size=72B cardinality=1.32K cost=577
| in pipelines: 42(GETNEXT), 46(OPEN)
|
-|--F25:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
-| | Per-Instance Resources: mem-estimate=14.69MB mem-reservation=3.88MB thread-reservation=1
-| | max-parallelism=120 segment-costs=[3970, 1319] cpu-comparison-result=120 [max(120 (self) vs 120 (sum children))]
+|--F25:PLAN FRAGMENT [RANDOM] hosts=10 instances=30 (adjusted from 120)
+| | Per-Instance Resources: mem-estimate=11.94MB mem-reservation=3.88MB thread-reservation=1
+| | max-parallelism=30 segment-costs=[3970, 1319] cpu-comparison-result=30 [max(30 (self) vs 22 (sum children))]
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: s_store_sk
@@ -976,19 +976,19 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | in pipelines: 46(GETNEXT), 07(OPEN)
| |
| 45:EXCHANGE [HASH(s_store_sk)]
-| | mem-estimate=4.69MB mem-reservation=0B thread-reservation=0
+| | mem-estimate=404.64KB mem-reservation=0B thread-reservation=0
| | tuple-ids=8 row-size=36B cardinality=1.32K cost=364
| | in pipelines: 07(GETNEXT)
| |
-| F04:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+| F04:PLAN FRAGMENT [RANDOM] hosts=10 instances=10 (adjusted from 120)
| Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
-| Per-Instance Resources: mem-estimate=44.75MB mem-reservation=2.50MB thread-reservation=1
-| max-parallelism=130 segment-costs=[1269007174, 3737] cpu-comparison-result=120 [max(120 (self) vs 22 (sum children))]
+| Per-Instance Resources: mem-estimate=30.69MB mem-reservation=2.50MB thread-reservation=1
+| max-parallelism=10 segment-costs=[20036633, 3737] cpu-comparison-result=22 [max(10 (self) vs 22 (sum children))]
| 12:AGGREGATE [STREAMING]
| | output: sum(sr_return_amt), sum(sr_net_loss)
| | group by: s_store_sk
| | mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=8 row-size=36B cardinality=1.32K cost=313608546
+| | tuple-ids=8 row-size=36B cardinality=1.32K cost=5257514
| | in pipelines: 07(GETNEXT)
| |
| 11:HASH JOIN [INNER JOIN, BROADCAST]
@@ -996,17 +996,17 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | hash predicates: sr_store_sk = s_store_sk
| | fk/pk conjuncts: sr_store_sk = s_store_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=5,6,7 row-size=28B cardinality=863.99M cost=378167824
+| | tuple-ids=5,6,7 row-size=28B cardinality=13.37M cost=5849901
| | in pipelines: 07(GETNEXT), 09(OPEN)
| |
| |--F26:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | Per-Instance Resources: mem-estimate=2.95MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
| | | max-parallelism=10 segment-costs=[3140]
| | JOIN BUILD
| | | join-table-id=01 plan-id=02 cohort-id=02
| | | build expressions: s_store_sk
| | | runtime filters: RF004[bloom] <- s_store_sk, RF005[min_max] <- s_store_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=1350
+| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=1350
| | |
| | 44:EXCHANGE [BROADCAST]
| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
@@ -1031,26 +1031,26 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | hash predicates: sr_returned_date_sk = d_date_sk
| | fk/pk conjuncts: sr_returned_date_sk = d_date_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=5,6 row-size=24B cardinality=863.99M cost=378167824
+| | tuple-ids=5,6 row-size=24B cardinality=13.37M cost=5849901
| | in pipelines: 07(GETNEXT), 08(OPEN)
| |
| |--F27:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[17005]
+| | | Per-Instance Resources: mem-estimate=2.95MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[71]
| | JOIN BUILD
| | | join-table-id=02 plan-id=03 cohort-id=02
| | | build expressions: d_date_sk
| | | runtime filters: RF006[bloom] <- d_date_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=31
| | |
| | 43:EXCHANGE [BROADCAST]
-| | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=6 row-size=8B cardinality=7.30K cost=9700
+| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=6 row-size=8B cardinality=31 cost=40
| | | in pipelines: 08(GETNEXT)
| | |
| | F05:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[12835]
+| | max-parallelism=1 segment-costs=[12521]
| | 08:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| | HDFS partitions=1/1 files=1 size=2.17MB
| | predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
@@ -1061,19 +1061,19 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=6 row-size=8B cardinality=7.30K cost=12520
+| | tuple-ids=6 row-size=8B cardinality=31 cost=12520
| | in pipelines: 08(GETNEXT)
| |
| 07:SCAN HDFS [tpcds_partitioned_parquet_snap.store_returns, RANDOM]
| HDFS partitions=2004/2004 files=2004 size=33.63MB
-| runtime filters: RF005[min_max] -> sr_store_sk, RF004[bloom] -> sr_store_sk, RF006[bloom] -> sr_returned_date_sk
+| runtime filters: RF005[min_max] -> sr_store_sk, RF006[bloom] -> sr_returned_date_sk, RF004[bloom] -> sr_store_sk
| stored statistics:
| table: rows=863.99M size=48.14GB
| partitions: 2004/2004 rows=863.99M
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=30.09M
+| extrapolated-rows=disabled max-scan-range-rows=30.09M est-scan-range=32(filtered from 2004)
| mem-estimate=16.00MB mem-reservation=512.00KB thread-reservation=0
-| tuple-ids=5 row-size=16B cardinality=863.99M cost=199062980
+| tuple-ids=5 row-size=16B cardinality=13.37M(filtered from 863.99M) cost=3079317
| in pipelines: 07(GETNEXT)
|
42:AGGREGATE [FINALIZE]
@@ -1084,19 +1084,19 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| in pipelines: 42(GETNEXT), 01(OPEN)
|
41:EXCHANGE [HASH(s_store_sk)]
-| mem-estimate=4.69MB mem-reservation=0B thread-reservation=0
+| mem-estimate=1.18MB mem-reservation=0B thread-reservation=0
| tuple-ids=3 row-size=36B cardinality=1.32K cost=364
| in pipelines: 01(GETNEXT)
|
-F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=30 (adjusted from 120)
Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
-Per-Instance Resources: mem-estimate=44.75MB mem-reservation=10.00MB thread-reservation=1
-max-parallelism=1270 segment-costs=[12686429509, 3737] cpu-comparison-result=120 [max(120 (self) vs 22 (sum children))]
+Per-Instance Resources: mem-estimate=30.69MB mem-reservation=10.00MB thread-reservation=1
+max-parallelism=30 segment-costs=[216019312, 3737] cpu-comparison-result=30 [max(30 (self) vs 22 (sum children))]
06:AGGREGATE [STREAMING]
| output: sum(ss_ext_sales_price), sum(ss_net_profit)
| group by: s_store_sk
| mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=3 row-size=36B cardinality=1.32K cost=3132389175
+| tuple-ids=3 row-size=36B cardinality=1.32K cost=53642531
| in pipelines: 01(GETNEXT)
|
05:HASH JOIN [INNER JOIN, BROADCAST]
@@ -1104,17 +1104,17 @@ max-parallelism=1270 segment-costs=[12686429509, 3737] cpu-comparison-result=120
| hash predicates: ss_store_sk = s_store_sk
| fk/pk conjuncts: ss_store_sk = s_store_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1,2 row-size=28B cardinality=8.64G cost=3781699633
+| tuple-ids=0,1,2 row-size=28B cardinality=146.84M cost=64272307
| in pipelines: 01(GETNEXT), 03(OPEN)
|
|--F33:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | Per-Instance Resources: mem-estimate=6.83MB mem-reservation=6.81MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[3140]
| JOIN BUILD
| | join-table-id=08 plan-id=09 cohort-id=01
| | build expressions: s_store_sk
| | runtime filters: RF000[bloom] <- s_store_sk, RF001[min_max] <- s_store_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=1350
+| | mem-estimate=5.81MB mem-reservation=5.81MB spill-buffer=64.00KB thread-reservation=0 cost=1350
| |
| 40:EXCHANGE [BROADCAST]
| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
@@ -1139,26 +1139,26 @@ max-parallelism=1270 segment-costs=[12686429509, 3737] cpu-comparison-result=120
| hash predicates: ss_sold_date_sk = d_date_sk
| fk/pk conjuncts: ss_sold_date_sk = d_date_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1 row-size=24B cardinality=8.64G cost=3781699633
+| tuple-ids=0,1 row-size=24B cardinality=146.84M cost=64272307
| in pipelines: 01(GETNEXT), 02(OPEN)
|
|--F34:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
+| | Per-Instance Resources: mem-estimate=6.83MB mem-reservation=6.81MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[71]
| JOIN BUILD
| | join-table-id=09 plan-id=10 cohort-id=01
| | build expressions: d_date_sk
| | runtime filters: RF002[bloom] <- d_date_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | mem-estimate=5.81MB mem-reservation=5.81MB spill-buffer=64.00KB thread-reservation=0 cost=31
| |
| 39:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=1 row-size=8B cardinality=7.30K cost=9700
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=1 row-size=8B cardinality=31 cost=40
| | in pipelines: 02(GETNEXT)
| |
| F01:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
+| max-parallelism=1 segment-costs=[12521]
| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| HDFS partitions=1/1 files=1 size=2.17MB
| predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
@@ -1169,26 +1169,26 @@ max-parallelism=1270 segment-costs=[12686429509, 3737] cpu-comparison-result=120
| parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=1 row-size=8B cardinality=7.30K cost=12520
+| tuple-ids=1 row-size=8B cardinality=31 cost=12520
| in pipelines: 02(GETNEXT)
|
01:SCAN HDFS [tpcds_partitioned_parquet_snap.store_sales, RANDOM]
HDFS partitions=1824/1824 files=1824 size=199.33MB
- runtime filters: RF001[min_max] -> ss_store_sk, RF000[bloom] -> ss_store_sk, RF002[bloom] -> ss_sold_date_sk
+ runtime filters: RF001[min_max] -> ss_store_sk, RF002[bloom] -> ss_sold_date_sk, RF000[bloom] -> ss_store_sk
stored statistics:
table: rows=8.64G size=389.90GB
partitions: 1824/1824 rows=8.64G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=390.22M
+ extrapolated-rows=disabled max-scan-range-rows=390.22M est-scan-range=32(filtered from 1824)
mem-estimate=16.00MB mem-reservation=8.00MB thread-reservation=0
- tuple-ids=0 row-size=16B cardinality=8.64G cost=1990641068
+ tuple-ids=0 row-size=16B cardinality=146.84M(filtered from 8.64G) cost=33832167
in pipelines: 01(GETNEXT)
---- PARALLELPLANS
-Max Per-Host Resource Reservation: Memory=646.58MB Threads=117
-Per-Host Resource Estimates: Memory=3.76GB
+Max Per-Host Resource Reservation: Memory=155.83MB Threads=41
+Per-Host Resource Estimates: Memory=723MB
F24:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=4.07MB mem-reservation=4.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[538] cpu-comparison-result=650 [max(1 (self) vs 650 (sum children))]
+| max-parallelism=1 segment-costs=[538] cpu-comparison-result=134 [max(1 (self) vs 134 (sum children))]
PLAN-ROOT SINK
| output exprs: CASE valid_tid(32,30,31) WHEN 30 THEN channel WHEN 31 THEN channel WHEN 32 THEN NULL END, CASE valid_tid(32,30,31) WHEN 30 THEN id WHEN 31 THEN NULL WHEN 32 THEN NULL END, aggif(valid_tid(32,30,31) IN (30, 31, 32), CASE valid_tid(32,30,31) WHEN 30 THEN sum(sales) WHEN 31 THEN sum(sales) WHEN 32 THEN sum(sales) END), aggif(valid_tid(32,30,31) IN (30, 31, 32), CASE valid_tid(32,30,31) WHEN 30 THEN sum(`returns`) WHEN 31 THEN sum(`returns`) WHEN 32 THEN sum(`returns`) END), [...]
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=500
@@ -1201,8 +1201,8 @@ PLAN-ROOT SINK
| in pipelines: 38(GETNEXT)
|
F23:PLAN FRAGMENT [HASH(CASE valid_tid(32,30,31) WHEN 30 THEN murmur_hash(channel) WHEN 31 THEN murmur_hash(channel) WHEN 32 THEN murmur_hash(NULL) END,CASE valid_tid(32,30,31) WHEN 30 THEN murmur_hash(id) WHEN 31 THEN murmur_hash(NULL) WHEN 32 THEN murmur_hash(NULL) END)] hosts=10 instances=10 (adjusted from 120)
-Per-Instance Resources: mem-estimate=54.04MB mem-reservation=7.75MB thread-reservation=1
-max-parallelism=10 segment-costs=[51386, 43669, 52232, 199] cpu-comparison-result=650 [max(10 (self) vs 650 (sum children))]
+Per-Instance Resources: mem-estimate=40.00MB mem-reservation=7.75MB thread-reservation=1
+max-parallelism=10 segment-costs=[51386, 43669, 52232, 199] cpu-comparison-result=134 [max(10 (self) vs 134 (sum children))]
38:TOP-N [LIMIT=100]
| order by: CASE valid_tid(32,30,31) WHEN 30 THEN channel WHEN 31 THEN channel WHEN 32 THEN NULL END ASC, CASE valid_tid(32,30,31) WHEN 30 THEN id WHEN 31 THEN NULL WHEN 32 THEN NULL END ASC
| mem-estimate=6.25KB mem-reservation=0B thread-reservation=0
@@ -1231,13 +1231,13 @@ max-parallelism=10 segment-costs=[51386, 43669, 52232, 199] cpu-comparison-resul
| in pipelines: 63(GETNEXT), 42(OPEN), 49(OPEN), 57(OPEN)
|
62:EXCHANGE [HASH(CASE valid_tid(32,30,31) WHEN 30 THEN murmur_hash(channel) WHEN 31 THEN murmur_hash(channel) WHEN 32 THEN murmur_hash(NULL) END,CASE valid_tid(32,30,31) WHEN 30 THEN murmur_hash(id) WHEN 31 THEN murmur_hash(NULL) WHEN 32 THEN murmur_hash(NULL) END)]
-| mem-estimate=24.04MB mem-reservation=0B thread-reservation=0
+| mem-estimate=6.11MB mem-reservation=0B thread-reservation=0
| tuple-ids=30N,31N,32N row-size=192B cardinality=7.23K cost=7717
| in pipelines: 42(GETNEXT), 49(GETNEXT), 57(GETNEXT)
|
-F22:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=52.67MB mem-reservation=7.94MB thread-reservation=1
-max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparison-result=650 [max(120 (self) vs 650 (sum children))]
+F22:PLAN FRAGMENT [RANDOM] hosts=10 instances=30 (adjusted from 120)
+Per-Instance Resources: mem-estimate=49.15MB mem-reservation=7.94MB thread-reservation=1
+max-parallelism=30 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparison-result=134 [max(30 (self) vs 134 (sum children))]
36:AGGREGATE [STREAMING]
| Class 0
| output: sum(sales), sum(returns), sum(profit)
@@ -1265,9 +1265,9 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | tuple-ids=21,26N row-size=72B cardinality=3.60K cost=1575
| | in pipelines: 57(GETNEXT), 61(OPEN)
| |
-| |--F30:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+| |--F30:PLAN FRAGMENT [RANDOM] hosts=10 instances=30 (adjusted from 120)
| | | Per-Instance Resources: mem-estimate=11.94MB mem-reservation=3.88MB thread-reservation=1
-| | | max-parallelism=120 segment-costs=[11374, 3600] cpu-comparison-result=120 [max(120 (self) vs 40 (sum children))]
+| | | max-parallelism=30 segment-costs=[11374, 3600] cpu-comparison-result=30 [max(30 (self) vs 22 (sum children))]
| | JOIN BUILD
| | | join-table-id=05 plan-id=06 cohort-id=01
| | | build expressions: wp_web_page_sk
@@ -1281,104 +1281,104 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | | in pipelines: 61(GETNEXT), 29(OPEN)
| | |
| | 60:EXCHANGE [HASH(wp_web_page_sk)]
-| | | mem-estimate=1.57MB mem-reservation=0B thread-reservation=0
+| | | mem-estimate=412.66KB mem-reservation=0B thread-reservation=0
| | | tuple-ids=26 row-size=36B cardinality=3.60K cost=996
| | | in pipelines: 29(GETNEXT)
| | |
-| | F18:PLAN FRAGMENT [RANDOM] hosts=10 instances=40 (adjusted from 120)
+| | F18:PLAN FRAGMENT [RANDOM] hosts=10 instances=10 (adjusted from 120)
| | Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
-| | Per-Instance Resources: mem-estimate=44.75MB mem-reservation=2.25MB thread-reservation=1
-| | max-parallelism=40 segment-costs=[313755974, 10200] cpu-comparison-result=40 [max(40 (self) vs 22 (sum children))]
+| | Per-Instance Resources: mem-estimate=30.69MB mem-reservation=2.25MB thread-reservation=1
+| | max-parallelism=10 segment-costs=[5747045, 10200] cpu-comparison-result=22 [max(10 (self) vs 22 (sum children))]
| | 34:AGGREGATE [STREAMING]
| | | output: sum(wr_return_amt), sum(wr_net_loss)
| | | group by: wp_web_page_sk
| | | mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=26 row-size=36B cardinality=3.60K cost=78101598
+| | | tuple-ids=26 row-size=36B cardinality=3.60K cost=2254148
| | | in pipelines: 29(GETNEXT)
| | |
| | 33:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash-table-id=06
-| | | hash predicates: wr_returned_date_sk = d_date_sk
-| | | fk/pk conjuncts: wr_returned_date_sk = d_date_sk
+| | | hash predicates: wr_web_page_sk = wp_web_page_sk
+| | | fk/pk conjuncts: wr_web_page_sk = wp_web_page_sk
| | | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=23,25,24 row-size=28B cardinality=212.35M cost=92943679
-| | | in pipelines: 29(GETNEXT), 30(OPEN)
+| | | tuple-ids=23,24,25 row-size=28B cardinality=3.11M cost=1376695
+| | | in pipelines: 29(GETNEXT), 31(OPEN)
| | |
| | |--F31:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | | Per-Instance Resources: mem-estimate=8.82MB mem-reservation=8.75MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | | max-parallelism=10 segment-costs=[17005]
+| | | | Per-Instance Resources: mem-estimate=2.96MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | | max-parallelism=10 segment-costs=[8380]
| | | JOIN BUILD
| | | | join-table-id=06 plan-id=07 cohort-id=04
-| | | | build expressions: d_date_sk
-| | | | runtime filters: RF016[bloom] <- d_date_sk
-| | | | mem-estimate=7.75MB mem-reservation=7.75MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | | | build expressions: wp_web_page_sk
+| | | | runtime filters: RF016[bloom] <- wp_web_page_sk, RF017[min_max] <- wp_web_page_sk
+| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=3600
| | | |
| | | 59:EXCHANGE [BROADCAST]
-| | | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | | tuple-ids=24 row-size=8B cardinality=7.30K cost=9700
-| | | | in pipelines: 30(GETNEXT)
+| | | | mem-estimate=22.06KB mem-reservation=0B thread-reservation=0
+| | | | tuple-ids=25 row-size=4B cardinality=3.60K cost=4780
+| | | | in pipelines: 31(GETNEXT)
| | | |
| | | F20:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | | max-parallelism=1 segment-costs=[12835]
-| | | 30:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| | | HDFS partitions=1/1 files=1 size=2.17MB
-| | | predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
+| | | Per-Instance Resources: mem-estimate=16.03MB mem-reservation=8.00KB thread-reservation=1
+| | | max-parallelism=1 segment-costs=[323]
+| | | 31:SCAN HDFS [tpcds_partitioned_parquet_snap.web_page, RANDOM]
+| | | HDFS partitions=1/1 files=1 size=5.45KB
| | | stored statistics:
-| | | table: rows=73.05K size=2.17MB
+| | | table: rows=3.60K size=72.76KB
| | | columns: all
-| | | extrapolated-rows=disabled max-scan-range-rows=73.05K
-| | | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
-| | | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
-| | | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | | tuple-ids=24 row-size=8B cardinality=7.30K cost=12520
-| | | in pipelines: 30(GETNEXT)
+| | | extrapolated-rows=disabled max-scan-range-rows=3.60K
+| | | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=0
+| | | tuple-ids=25 row-size=4B cardinality=3.60K cost=207
+| | | in pipelines: 31(GETNEXT)
| | |
| | 32:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash-table-id=07
-| | | hash predicates: wr_web_page_sk = wp_web_page_sk
-| | | fk/pk conjuncts: wr_web_page_sk = wp_web_page_sk
+| | | hash predicates: wr_returned_date_sk = d_date_sk
+| | | fk/pk conjuncts: wr_returned_date_sk = d_date_sk
| | | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=23,25 row-size=20B cardinality=212.35M cost=92943679
-| | | in pipelines: 29(GETNEXT), 31(OPEN)
+| | | tuple-ids=23,24 row-size=24B cardinality=3.17M cost=1386412
+| | | in pipelines: 29(GETNEXT), 30(OPEN)
| | |
| | |--F32:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | | Per-Instance Resources: mem-estimate=8.77MB mem-reservation=8.75MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | | max-parallelism=10 segment-costs=[8380]
+| | | | Per-Instance Resources: mem-estimate=2.95MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | | max-parallelism=10 segment-costs=[71]
| | | JOIN BUILD
| | | | join-table-id=07 plan-id=08 cohort-id=04
-| | | | build expressions: wp_web_page_sk
-| | | | runtime filters: RF018[bloom] <- wp_web_page_sk, RF019[min_max] <- wp_web_page_sk
-| | | | mem-estimate=7.75MB mem-reservation=7.75MB spill-buffer=64.00KB thread-reservation=0 cost=3600
+| | | | build expressions: d_date_sk
+| | | | runtime filters: RF018[bloom] <- d_date_sk
+| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=31
| | | |
| | | 58:EXCHANGE [BROADCAST]
-| | | | mem-estimate=22.06KB mem-reservation=0B thread-reservation=0
-| | | | tuple-ids=25 row-size=4B cardinality=3.60K cost=4780
-| | | | in pipelines: 31(GETNEXT)
+| | | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | | tuple-ids=24 row-size=8B cardinality=31 cost=40
+| | | | in pipelines: 30(GETNEXT)
| | | |
| | | F19:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | | Per-Instance Resources: mem-estimate=16.03MB mem-reservation=8.00KB thread-reservation=1
-| | | max-parallelism=1 segment-costs=[323]
-| | | 31:SCAN HDFS [tpcds_partitioned_parquet_snap.web_page, RANDOM]
-| | | HDFS partitions=1/1 files=1 size=5.45KB
+| | | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| | | max-parallelism=1 segment-costs=[12521]
+| | | 30:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| | | HDFS partitions=1/1 files=1 size=2.17MB
+| | | predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | | stored statistics:
-| | | table: rows=3.60K size=72.76KB
+| | | table: rows=73.05K size=2.17MB
| | | columns: all
-| | | extrapolated-rows=disabled max-scan-range-rows=3.60K
-| | | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=0
-| | | tuple-ids=25 row-size=4B cardinality=3.60K cost=207
-| | | in pipelines: 31(GETNEXT)
+| | | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
+| | | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
+| | | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | | tuple-ids=24 row-size=8B cardinality=31 cost=12520
+| | | in pipelines: 30(GETNEXT)
| | |
| | 29:SCAN HDFS [tpcds_partitioned_parquet_snap.web_returns, RANDOM]
| | HDFS partitions=2114/2114 files=2114 size=20.70MB
-| | runtime filters: RF019[min_max] -> wr_web_page_sk, RF018[bloom] -> wr_web_page_sk, RF016[bloom] -> wr_returned_date_sk
+| | runtime filters: RF017[min_max] -> wr_web_page_sk, RF018[bloom] -> wr_returned_date_sk, RF016[bloom] -> wr_web_page_sk
| | stored statistics:
| | table: rows=216.00M size=16.74GB
| | partitions: 2114/2114 rows=216.00M
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=9.41M
+| | extrapolated-rows=disabled max-scan-range-rows=9.41M est-scan-range=32(filtered from 2114)
| | mem-estimate=16.00MB mem-reservation=256.00KB thread-reservation=0
-| | tuple-ids=23 row-size=16B cardinality=212.35M(filtered from 216.00M) cost=49767018
+| | tuple-ids=23 row-size=16B cardinality=3.17M(filtered from 216.00M) cost=729790
| | in pipelines: 29(GETNEXT)
| |
| 57:AGGREGATE [FINALIZE]
@@ -1389,104 +1389,104 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | in pipelines: 57(GETNEXT), 23(OPEN)
| |
| 56:EXCHANGE [HASH(wp_web_page_sk)]
-| | mem-estimate=4.70MB mem-reservation=0B thread-reservation=0
+| | mem-estimate=412.66KB mem-reservation=0B thread-reservation=0
| | tuple-ids=21 row-size=36B cardinality=3.60K cost=996
| | in pipelines: 23(GETNEXT)
| |
-| F14:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+| F14:PLAN FRAGMENT [RANDOM] hosts=10 instances=10 (adjusted from 120)
| Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
-| Per-Instance Resources: mem-estimate=44.75MB mem-reservation=2.12MB thread-reservation=1
-| max-parallelism=320 segment-costs=[3127337722, 10200] cpu-comparison-result=120 [max(120 (self) vs 22 (sum children))]
+| Per-Instance Resources: mem-estimate=30.69MB mem-reservation=2.12MB thread-reservation=1
+| max-parallelism=10 segment-costs=[54689653, 10200] cpu-comparison-result=22 [max(10 (self) vs 22 (sum children))]
| 28:AGGREGATE [STREAMING]
| | output: sum(ws_ext_sales_price), sum(ws_net_profit)
| | group by: wp_web_page_sk
| | mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=21 row-size=36B cardinality=3.60K cost=770858205
+| | tuple-ids=21 row-size=36B cardinality=3.60K cost=14208376
| | in pipelines: 23(GETNEXT)
| |
| 27:HASH JOIN [INNER JOIN, BROADCAST]
| | hash-table-id=11
-| | hash predicates: ws_sold_date_sk = d_date_sk
-| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
+| | hash predicates: ws_web_page_sk = wp_web_page_sk
+| | fk/pk conjuncts: ws_web_page_sk = wp_web_page_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=18,20,19 row-size=28B cardinality=2.12G cost=929411449
-| | in pipelines: 23(GETNEXT), 24(OPEN)
+| | tuple-ids=18,19,20 row-size=28B cardinality=36.09M cost=15955339
+| | in pipelines: 23(GETNEXT), 25(OPEN)
| |
| |--F36:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[17005]
+| | | Per-Instance Resources: mem-estimate=2.96MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[8380]
| | JOIN BUILD
| | | join-table-id=11 plan-id=12 cohort-id=01
-| | | build expressions: d_date_sk
-| | | runtime filters: RF012[bloom] <- d_date_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | | build expressions: wp_web_page_sk
+| | | runtime filters: RF012[bloom] <- wp_web_page_sk, RF013[min_max] <- wp_web_page_sk
+| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=3600
| | |
| | 55:EXCHANGE [BROADCAST]
-| | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=19 row-size=8B cardinality=7.30K cost=9700
-| | | in pipelines: 24(GETNEXT)
+| | | mem-estimate=22.06KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=20 row-size=4B cardinality=3.60K cost=4780
+| | | in pipelines: 25(GETNEXT)
| | |
| | F16:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[12835]
-| | 24:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| | HDFS partitions=1/1 files=1 size=2.17MB
-| | predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
+| | Per-Instance Resources: mem-estimate=16.03MB mem-reservation=8.00KB thread-reservation=1
+| | max-parallelism=1 segment-costs=[323]
+| | 25:SCAN HDFS [tpcds_partitioned_parquet_snap.web_page, RANDOM]
+| | HDFS partitions=1/1 files=1 size=5.45KB
| | stored statistics:
-| | table: rows=73.05K size=2.17MB
+| | table: rows=3.60K size=72.76KB
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=73.05K
-| | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
-| | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
-| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=19 row-size=8B cardinality=7.30K cost=12520
-| | in pipelines: 24(GETNEXT)
+| | extrapolated-rows=disabled max-scan-range-rows=3.60K
+| | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=0
+| | tuple-ids=20 row-size=4B cardinality=3.60K cost=207
+| | in pipelines: 25(GETNEXT)
| |
| 26:HASH JOIN [INNER JOIN, BROADCAST]
| | hash-table-id=12
-| | hash predicates: ws_web_page_sk = wp_web_page_sk
-| | fk/pk conjuncts: ws_web_page_sk = wp_web_page_sk
+| | hash predicates: ws_sold_date_sk = d_date_sk
+| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=18,20 row-size=20B cardinality=2.12G cost=929411449
-| | in pipelines: 23(GETNEXT), 25(OPEN)
+| | tuple-ids=18,19 row-size=24B cardinality=36.71M cost=16067959
+| | in pipelines: 23(GETNEXT), 24(OPEN)
| |
| |--F37:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[8380]
+| | | Per-Instance Resources: mem-estimate=2.95MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[71]
| | JOIN BUILD
| | | join-table-id=12 plan-id=13 cohort-id=01
-| | | build expressions: wp_web_page_sk
-| | | runtime filters: RF014[bloom] <- wp_web_page_sk, RF015[min_max] <- wp_web_page_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=3600
+| | | build expressions: d_date_sk
+| | | runtime filters: RF014[bloom] <- d_date_sk
+| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=31
| | |
| | 54:EXCHANGE [BROADCAST]
-| | | mem-estimate=22.06KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=20 row-size=4B cardinality=3.60K cost=4780
-| | | in pipelines: 25(GETNEXT)
+| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=19 row-size=8B cardinality=31 cost=40
+| | | in pipelines: 24(GETNEXT)
| | |
| | F15:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | Per-Instance Resources: mem-estimate=16.03MB mem-reservation=8.00KB thread-reservation=1
-| | max-parallelism=1 segment-costs=[323]
-| | 25:SCAN HDFS [tpcds_partitioned_parquet_snap.web_page, RANDOM]
-| | HDFS partitions=1/1 files=1 size=5.45KB
+| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| | max-parallelism=1 segment-costs=[12521]
+| | 24:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| | HDFS partitions=1/1 files=1 size=2.17MB
+| | predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | stored statistics:
-| | table: rows=3.60K size=72.76KB
+| | table: rows=73.05K size=2.17MB
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=3.60K
-| | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=0
-| | tuple-ids=20 row-size=4B cardinality=3.60K cost=207
-| | in pipelines: 25(GETNEXT)
+| | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
+| | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
+| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | tuple-ids=19 row-size=8B cardinality=31 cost=12520
+| | in pipelines: 24(GETNEXT)
| |
| 23:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales, RANDOM]
| HDFS partitions=1824/1824 files=1824 size=84.18MB
-| runtime filters: RF015[min_max] -> ws_web_page_sk, RF014[bloom] -> ws_web_page_sk, RF012[bloom] -> ws_sold_date_sk
+| runtime filters: RF013[min_max] -> ws_web_page_sk, RF014[bloom] -> ws_sold_date_sk, RF012[bloom] -> ws_web_page_sk
| stored statistics:
| table: rows=2.16G size=145.75GB
| partitions: 1824/1824 rows=2.16G
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=3.37M
+| extrapolated-rows=disabled max-scan-range-rows=3.37M est-scan-range=32(filtered from 1824)
| mem-estimate=16.00MB mem-reservation=128.00KB thread-reservation=0
-| tuple-ids=18 row-size=16B cardinality=2.12G(filtered from 2.16G) cost=497656619
+| tuple-ids=18 row-size=16B cardinality=36.71M(filtered from 2.16G) cost=8457979
| in pipelines: 23(GETNEXT)
|
|--22:NESTED LOOP JOIN [CROSS JOIN, BROADCAST]
@@ -1497,7 +1497,7 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| |
| |--F28:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
| | | Per-Instance Resources: mem-estimate=22.12KB mem-reservation=0B thread-reservation=1
-| | | max-parallelism=10 segment-costs=[60] cpu-comparison-result=50 [max(10 (self) vs 50 (sum children))]
+| | | max-parallelism=10 segment-costs=[60] cpu-comparison-result=11 [max(10 (self) vs 11 (sum children))]
| | JOIN BUILD
| | | join-table-id=03 plan-id=04 cohort-id=01
| | | mem-estimate=1.69KB mem-reservation=0B thread-reservation=0 cost=0
@@ -1509,7 +1509,7 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | |
| | F13:PLAN FRAGMENT [HASH(cr_call_center_sk)] hosts=10 instances=10 (adjusted from 120)
| | Per-Instance Resources: mem-estimate=10.16MB mem-reservation=1.94MB thread-reservation=1
-| | max-parallelism=10 segment-costs=[155, 5] cpu-comparison-result=50 [max(10 (self) vs 50 (sum children))]
+| | max-parallelism=10 segment-costs=[155, 5] cpu-comparison-result=11 [max(10 (self) vs 11 (sum children))]
| | 52:AGGREGATE [FINALIZE]
| | | output: sum:merge(cr_return_amount), sum:merge(cr_net_loss)
| | | group by: cr_call_center_sk
@@ -1518,19 +1518,19 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | | in pipelines: 52(GETNEXT), 18(OPEN)
| | |
| | 51:EXCHANGE [HASH(cr_call_center_sk)]
-| | | mem-estimate=93.92KB mem-reservation=0B thread-reservation=0
+| | | mem-estimate=18.92KB mem-reservation=0B thread-reservation=0
| | | tuple-ids=16 row-size=36B cardinality=48 cost=13
| | | in pipelines: 18(GETNEXT)
| | |
-| | F11:PLAN FRAGMENT [RANDOM] hosts=10 instances=50 (adjusted from 120)
+| | F11:PLAN FRAGMENT [RANDOM] hosts=10 instances=10 (adjusted from 120)
| | Per-Host Shared Resources: mem-estimate=1.00MB mem-reservation=1.00MB thread-reservation=0 runtime-filters-memory=1.00MB
| | Per-Instance Resources: mem-estimate=27.56MB mem-reservation=2.03MB thread-reservation=1
-| | max-parallelism=50 segment-costs=[445251726, 136] cpu-comparison-result=50 [max(50 (self) vs 11 (sum children))]
+| | max-parallelism=10 segment-costs=[6715181, 136] cpu-comparison-result=11 [max(10 (self) vs 11 (sum children))]
| | 21:AGGREGATE [STREAMING]
| | | output: sum(cr_return_amount), sum(cr_net_loss)
| | | group by: cr_call_center_sk
| | | mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=16 row-size=36B cardinality=48 cost=156621176
+| | | tuple-ids=16 row-size=36B cardinality=48 cost=2371712
| | | in pipelines: 18(GETNEXT)
| | |
| | 20:HASH JOIN [INNER JOIN, BROADCAST]
@@ -1538,26 +1538,26 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | | hash predicates: cr_returned_date_sk = d_date_sk
| | | fk/pk conjuncts: cr_returned_date_sk = d_date_sk
| | | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | | tuple-ids=14,15 row-size=24B cardinality=432.02M cost=189093836
+| | | tuple-ids=14,15 row-size=24B cardinality=6.50M cost=2845587
| | | in pipelines: 18(GETNEXT), 19(OPEN)
| | |
| | |--F29:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | | Per-Instance Resources: mem-estimate=10.75MB mem-reservation=10.69MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | | max-parallelism=10 segment-costs=[17005]
+| | | | Per-Instance Resources: mem-estimate=2.95MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | | max-parallelism=10 segment-costs=[71]
| | | JOIN BUILD
| | | | join-table-id=04 plan-id=05 cohort-id=03
| | | | build expressions: d_date_sk
| | | | runtime filters: RF010[bloom] <- d_date_sk
-| | | | mem-estimate=9.69MB mem-reservation=9.69MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=31
| | | |
| | | 50:EXCHANGE [BROADCAST]
-| | | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | | tuple-ids=15 row-size=8B cardinality=7.30K cost=9700
+| | | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | | tuple-ids=15 row-size=8B cardinality=31 cost=40
| | | | in pipelines: 19(GETNEXT)
| | | |
| | | F12:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| | | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | | max-parallelism=1 segment-costs=[12835]
+| | | max-parallelism=1 segment-costs=[12521]
| | | 19:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| | | HDFS partitions=1/1 files=1 size=2.17MB
| | | predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
@@ -1568,7 +1568,7 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | | tuple-ids=15 row-size=8B cardinality=7.30K cost=12520
+| | | tuple-ids=15 row-size=8B cardinality=31 cost=12520
| | | in pipelines: 19(GETNEXT)
| | |
| | 18:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_returns, RANDOM]
@@ -1578,9 +1578,9 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | table: rows=432.02M size=32.77GB
| | partitions: 2060/2060 rows=432.02M
| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=416.82K
+| | extrapolated-rows=disabled max-scan-range-rows=416.82K est-scan-range=32(filtered from 2060)
| | mem-estimate=16.00MB mem-reservation=32.00KB thread-reservation=0
-| | tuple-ids=14 row-size=16B cardinality=432.02M cost=99536714
+| | tuple-ids=14 row-size=16B cardinality=6.50M(filtered from 432.02M) cost=1497882
| | in pipelines: 18(GETNEXT)
| |
| 49:AGGREGATE [FINALIZE]
@@ -1591,19 +1591,19 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | in pipelines: 49(GETNEXT), 14(OPEN)
| |
| 48:EXCHANGE [HASH(cs_call_center_sk)]
-| | mem-estimate=225.17KB mem-reservation=0B thread-reservation=0
+| | mem-estimate=18.92KB mem-reservation=0B thread-reservation=0
| | tuple-ids=12 row-size=36B cardinality=48 cost=13
| | in pipelines: 14(GETNEXT)
| |
-| F08:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+| F08:PLAN FRAGMENT [RANDOM] hosts=10 instances=10 (adjusted from 120)
| Per-Host Shared Resources: mem-estimate=1.00MB mem-reservation=1.00MB thread-reservation=0 runtime-filters-memory=1.00MB
-| Per-Instance Resources: mem-estimate=44.75MB mem-reservation=3.00MB thread-reservation=1
-| max-parallelism=450 segment-costs=[4452287381, 136] cpu-comparison-result=120 [max(120 (self) vs 11 (sum children))]
+| Per-Instance Resources: mem-estimate=30.69MB mem-reservation=3.00MB thread-reservation=1
+| max-parallelism=10 segment-costs=[75394832, 136] cpu-comparison-result=11 [max(10 (self) vs 11 (sum children))]
| 17:AGGREGATE [STREAMING]
| | output: sum(cs_ext_sales_price), sum(cs_net_profit)
| | group by: cs_call_center_sk
| | mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=12 row-size=36B cardinality=48 cost=1566043284
+| | tuple-ids=12 row-size=36B cardinality=48 cost=26528876
| | in pipelines: 14(GETNEXT)
| |
| 16:HASH JOIN [INNER JOIN, BROADCAST]
@@ -1611,26 +1611,26 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | hash predicates: cs_sold_date_sk = d_date_sk
| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=10,11 row-size=24B cardinality=4.32G cost=1890898131
+| | tuple-ids=10,11 row-size=24B cardinality=73.14M cost=32014113
| | in pipelines: 14(GETNEXT), 15(OPEN)
| |
| |--F35:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[17005]
+| | | Per-Instance Resources: mem-estimate=2.95MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[71]
| | JOIN BUILD
| | | join-table-id=10 plan-id=11 cohort-id=01
| | | build expressions: d_date_sk
| | | runtime filters: RF008[bloom] <- d_date_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=31
| | |
| | 47:EXCHANGE [BROADCAST]
-| | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=11 row-size=8B cardinality=7.30K cost=9700
+| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=11 row-size=8B cardinality=31 cost=40
| | | in pipelines: 15(GETNEXT)
| | |
| | F09:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[12835]
+| | max-parallelism=1 segment-costs=[12521]
| | 15:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| | HDFS partitions=1/1 files=1 size=2.17MB
| | predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
@@ -1641,7 +1641,7 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=11 row-size=8B cardinality=7.30K cost=12520
+| | tuple-ids=11 row-size=8B cardinality=31 cost=12520
| | in pipelines: 15(GETNEXT)
| |
| 14:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales, RANDOM]
@@ -1651,9 +1651,9 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| table: rows=4.32G size=280.96GB
| partitions: 1831/1831 rows=4.32G
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=21.52M
+| extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=31(filtered from 1831)
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=10 row-size=16B cardinality=4.32G cost=995345966
+| tuple-ids=10 row-size=16B cardinality=73.14M(filtered from 4.32G) cost=16851843
| in pipelines: 14(GETNEXT)
|
13:HASH JOIN [LEFT OUTER JOIN, PARTITIONED]
@@ -1664,9 +1664,9 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| tuple-ids=3,8N row-size=72B cardinality=1.32K cost=577
| in pipelines: 42(GETNEXT), 46(OPEN)
|
-|--F25:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
-| | Per-Instance Resources: mem-estimate=14.69MB mem-reservation=3.88MB thread-reservation=1
-| | max-parallelism=120 segment-costs=[3970, 1319] cpu-comparison-result=120 [max(120 (self) vs 120 (sum children))]
+|--F25:PLAN FRAGMENT [RANDOM] hosts=10 instances=30 (adjusted from 120)
+| | Per-Instance Resources: mem-estimate=11.94MB mem-reservation=3.88MB thread-reservation=1
+| | max-parallelism=30 segment-costs=[3970, 1319] cpu-comparison-result=30 [max(30 (self) vs 22 (sum children))]
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: s_store_sk
@@ -1680,19 +1680,19 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | in pipelines: 46(GETNEXT), 07(OPEN)
| |
| 45:EXCHANGE [HASH(s_store_sk)]
-| | mem-estimate=4.69MB mem-reservation=0B thread-reservation=0
+| | mem-estimate=404.64KB mem-reservation=0B thread-reservation=0
| | tuple-ids=8 row-size=36B cardinality=1.32K cost=364
| | in pipelines: 07(GETNEXT)
| |
-| F04:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+| F04:PLAN FRAGMENT [RANDOM] hosts=10 instances=10 (adjusted from 120)
| Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
-| Per-Instance Resources: mem-estimate=44.75MB mem-reservation=2.50MB thread-reservation=1
-| max-parallelism=130 segment-costs=[1269007174, 3737] cpu-comparison-result=120 [max(120 (self) vs 22 (sum children))]
+| Per-Instance Resources: mem-estimate=30.69MB mem-reservation=2.50MB thread-reservation=1
+| max-parallelism=10 segment-costs=[20036633, 3737] cpu-comparison-result=22 [max(10 (self) vs 22 (sum children))]
| 12:AGGREGATE [STREAMING]
| | output: sum(sr_return_amt), sum(sr_net_loss)
| | group by: s_store_sk
| | mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=8 row-size=36B cardinality=1.32K cost=313608546
+| | tuple-ids=8 row-size=36B cardinality=1.32K cost=5257514
| | in pipelines: 07(GETNEXT)
| |
| 11:HASH JOIN [INNER JOIN, BROADCAST]
@@ -1700,17 +1700,17 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | hash predicates: sr_store_sk = s_store_sk
| | fk/pk conjuncts: sr_store_sk = s_store_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=5,6,7 row-size=28B cardinality=863.99M cost=378167824
+| | tuple-ids=5,6,7 row-size=28B cardinality=13.37M cost=5849901
| | in pipelines: 07(GETNEXT), 09(OPEN)
| |
| |--F26:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | Per-Instance Resources: mem-estimate=2.95MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
| | | max-parallelism=10 segment-costs=[3140]
| | JOIN BUILD
| | | join-table-id=01 plan-id=02 cohort-id=02
| | | build expressions: s_store_sk
| | | runtime filters: RF004[bloom] <- s_store_sk, RF005[min_max] <- s_store_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=1350
+| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=1350
| | |
| | 44:EXCHANGE [BROADCAST]
| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
@@ -1735,26 +1735,26 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | hash predicates: sr_returned_date_sk = d_date_sk
| | fk/pk conjuncts: sr_returned_date_sk = d_date_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=5,6 row-size=24B cardinality=863.99M cost=378167824
+| | tuple-ids=5,6 row-size=24B cardinality=13.37M cost=5849901
| | in pipelines: 07(GETNEXT), 08(OPEN)
| |
| |--F27:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | | max-parallelism=10 segment-costs=[17005]
+| | | Per-Instance Resources: mem-estimate=2.95MB mem-reservation=2.94MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[71]
| | JOIN BUILD
| | | join-table-id=02 plan-id=03 cohort-id=02
| | | build expressions: d_date_sk
| | | runtime filters: RF006[bloom] <- d_date_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0 cost=31
| | |
| | 43:EXCHANGE [BROADCAST]
-| | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=6 row-size=8B cardinality=7.30K cost=9700
+| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=6 row-size=8B cardinality=31 cost=40
| | | in pipelines: 08(GETNEXT)
| | |
| | F05:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[12835]
+| | max-parallelism=1 segment-costs=[12521]
| | 08:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| | HDFS partitions=1/1 files=1 size=2.17MB
| | predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
@@ -1765,19 +1765,19 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| | parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=6 row-size=8B cardinality=7.30K cost=12520
+| | tuple-ids=6 row-size=8B cardinality=31 cost=12520
| | in pipelines: 08(GETNEXT)
| |
| 07:SCAN HDFS [tpcds_partitioned_parquet_snap.store_returns, RANDOM]
| HDFS partitions=2004/2004 files=2004 size=33.63MB
-| runtime filters: RF005[min_max] -> sr_store_sk, RF004[bloom] -> sr_store_sk, RF006[bloom] -> sr_returned_date_sk
+| runtime filters: RF005[min_max] -> sr_store_sk, RF006[bloom] -> sr_returned_date_sk, RF004[bloom] -> sr_store_sk
| stored statistics:
| table: rows=863.99M size=48.14GB
| partitions: 2004/2004 rows=863.99M
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=30.09M
+| extrapolated-rows=disabled max-scan-range-rows=30.09M est-scan-range=32(filtered from 2004)
| mem-estimate=16.00MB mem-reservation=512.00KB thread-reservation=0
-| tuple-ids=5 row-size=16B cardinality=863.99M cost=199062980
+| tuple-ids=5 row-size=16B cardinality=13.37M(filtered from 863.99M) cost=3079317
| in pipelines: 07(GETNEXT)
|
42:AGGREGATE [FINALIZE]
@@ -1788,19 +1788,19 @@ max-parallelism=120 segment-costs=[3970, 155, 11374, 71422, 100530] cpu-comparis
| in pipelines: 42(GETNEXT), 01(OPEN)
|
41:EXCHANGE [HASH(s_store_sk)]
-| mem-estimate=4.69MB mem-reservation=0B thread-reservation=0
+| mem-estimate=1.18MB mem-reservation=0B thread-reservation=0
| tuple-ids=3 row-size=36B cardinality=1.32K cost=364
| in pipelines: 01(GETNEXT)
|
-F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=30 (adjusted from 120)
Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
-Per-Instance Resources: mem-estimate=44.75MB mem-reservation=10.00MB thread-reservation=1
-max-parallelism=1270 segment-costs=[12686429509, 3737] cpu-comparison-result=120 [max(120 (self) vs 22 (sum children))]
+Per-Instance Resources: mem-estimate=30.69MB mem-reservation=10.00MB thread-reservation=1
+max-parallelism=30 segment-costs=[216019312, 3737] cpu-comparison-result=30 [max(30 (self) vs 22 (sum children))]
06:AGGREGATE [STREAMING]
| output: sum(ss_ext_sales_price), sum(ss_net_profit)
| group by: s_store_sk
| mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=3 row-size=36B cardinality=1.32K cost=3132389175
+| tuple-ids=3 row-size=36B cardinality=1.32K cost=53642531
| in pipelines: 01(GETNEXT)
|
05:HASH JOIN [INNER JOIN, BROADCAST]
@@ -1808,17 +1808,17 @@ max-parallelism=1270 segment-costs=[12686429509, 3737] cpu-comparison-result=120
| hash predicates: ss_store_sk = s_store_sk
| fk/pk conjuncts: ss_store_sk = s_store_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1,2 row-size=28B cardinality=8.64G cost=3781699633
+| tuple-ids=0,1,2 row-size=28B cardinality=146.84M cost=64272307
| in pipelines: 01(GETNEXT), 03(OPEN)
|
|--F33:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | Per-Instance Resources: mem-estimate=6.83MB mem-reservation=6.81MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[3140]
| JOIN BUILD
| | join-table-id=08 plan-id=09 cohort-id=01
| | build expressions: s_store_sk
| | runtime filters: RF000[bloom] <- s_store_sk, RF001[min_max] <- s_store_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=1350
+| | mem-estimate=5.81MB mem-reservation=5.81MB spill-buffer=64.00KB thread-reservation=0 cost=1350
| |
| 40:EXCHANGE [BROADCAST]
| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
@@ -1843,26 +1843,26 @@ max-parallelism=1270 segment-costs=[12686429509, 3737] cpu-comparison-result=120
| hash predicates: ss_sold_date_sk = d_date_sk
| fk/pk conjuncts: ss_sold_date_sk = d_date_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1 row-size=24B cardinality=8.64G cost=3781699633
+| tuple-ids=0,1 row-size=24B cardinality=146.84M cost=64272307
| in pipelines: 01(GETNEXT), 02(OPEN)
|
|--F34:PLAN FRAGMENT [RANDOM] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
+| | Per-Instance Resources: mem-estimate=6.83MB mem-reservation=6.81MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[71]
| JOIN BUILD
| | join-table-id=09 plan-id=10 cohort-id=01
| | build expressions: d_date_sk
| | runtime filters: RF002[bloom] <- d_date_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | mem-estimate=5.81MB mem-reservation=5.81MB spill-buffer=64.00KB thread-reservation=0 cost=31
| |
| 39:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=1 row-size=8B cardinality=7.30K cost=9700
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=1 row-size=8B cardinality=31 cost=40
| | in pipelines: 02(GETNEXT)
| |
| F01:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
+| max-parallelism=1 segment-costs=[12521]
| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| HDFS partitions=1/1 files=1 size=2.17MB
| predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
@@ -1873,18 +1873,18 @@ max-parallelism=1270 segment-costs=[12686429509, 3737] cpu-comparison-result=120
| parquet statistics predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| parquet dictionary predicates: d_date <= DATE '2001-09-26', d_date >= DATE '2001-08-27'
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=1 row-size=8B cardinality=7.30K cost=12520
+| tuple-ids=1 row-size=8B cardinality=31 cost=12520
| in pipelines: 02(GETNEXT)
|
01:SCAN HDFS [tpcds_partitioned_parquet_snap.store_sales, RANDOM]
HDFS partitions=1824/1824 files=1824 size=199.33MB
- runtime filters: RF001[min_max] -> ss_store_sk, RF000[bloom] -> ss_store_sk, RF002[bloom] -> ss_sold_date_sk
+ runtime filters: RF001[min_max] -> ss_store_sk, RF002[bloom] -> ss_sold_date_sk, RF000[bloom] -> ss_store_sk
stored statistics:
table: rows=8.64G size=389.90GB
partitions: 1824/1824 rows=8.64G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=390.22M
+ extrapolated-rows=disabled max-scan-range-rows=390.22M est-scan-range=32(filtered from 1824)
mem-estimate=16.00MB mem-reservation=8.00MB thread-reservation=0
- tuple-ids=0 row-size=16B cardinality=8.64G cost=1990641068
+ tuple-ids=0 row-size=16B cardinality=146.84M(filtered from 8.64G) cost=33832167
in pipelines: 01(GETNEXT)
====
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q80.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q80.test
index 1b3acbb64..536ee980d 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q80.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q80.test
@@ -100,7 +100,7 @@ Max Per-Host Resource Reservation: Memory=75.11MB Threads=1
Per-Host Resource Estimates: Memory=40.23GB
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=40.23GB mem-reservation=75.11MB thread-reservation=1 runtime-filters-memory=10.00MB
-| max-parallelism=1 segment-costs=[8751684042, 4376161976, 2184339208, 342336, 221854, 313895, 500]
+| max-parallelism=1 segment-costs=[2907420430, 1453514230, 726896356, 342336, 221854, 313895, 500]
PLAN-ROOT SINK
| output exprs: CASE valid_tid(26,27,28) WHEN 26 THEN channel WHEN 27 THEN channel WHEN 28 THEN NULL END, CASE valid_tid(26,27,28) WHEN 26 THEN id WHEN 27 THEN NULL WHEN 28 THEN NULL END, aggif(valid_tid(26,27,28) IN (26, 27, 28), CASE valid_tid(26,27,28) WHEN 26 THEN sum(sales) WHEN 27 THEN sum(sales) WHEN 28 THEN sum(sales) END), aggif(valid_tid(26,27,28) IN (26, 27, 28), CASE valid_tid(26,27,28) WHEN 26 THEN sum(`returns`) WHEN 27 THEN sum(`returns`) WHEN 28 THEN sum(`returns`) END), [...]
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=500
@@ -141,35 +141,15 @@ PLAN-ROOT SINK
| | output: sum(ws_ext_sales_price), sum(coalesce(wr_return_amt, CAST(0 AS DECIMAL(7,2)))), sum(ws_net_profit - coalesce(wr_net_loss, CAST(0 AS DECIMAL(7,2))))
| | group by: web_site_id
| | mem-estimate=10.00MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=22 row-size=76B cardinality=33 cost=90905324
+| | tuple-ids=22 row-size=76B cardinality=33 cost=1545076
| | in pipelines: 36(GETNEXT), 25(OPEN)
| |
| 35:HASH JOIN [INNER JOIN]
-| | hash predicates: ws_sold_date_sk = d_date_sk
-| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=16,17N,20,19,21,18 row-size=117B cardinality=210.19M cost=92005507
-| | in pipelines: 25(GETNEXT), 27(OPEN)
-| |
-| |--27:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
-| | HDFS partitions=1/1 files=1 size=2.17MB
-| | predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | stored statistics:
-| | table: rows=73.05K size=2.17MB
-| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=73.05K
-| | parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=18 row-size=8B cardinality=7.30K cost=12520
-| | in pipelines: 27(GETNEXT)
-| |
-| 34:HASH JOIN [INNER JOIN]
| | hash predicates: ws_promo_sk = p_promo_sk
| | fk/pk conjuncts: ws_promo_sk = p_promo_sk
-| | runtime filters: RF018[bloom] <- p_promo_sk, RF019[min_max] <- p_promo_sk
+| | runtime filters: RF016[bloom] <- p_promo_sk, RF017[min_max] <- p_promo_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=16,17N,20,19,21 row-size=109B cardinality=210.19M cost=92636386
+| | tuple-ids=16,17N,18,20,19,21 row-size=117B cardinality=3.57M cost=1576161
| | in pipelines: 25(GETNEXT), 30(OPEN)
| |
| |--30:SCAN HDFS [tpcds_partitioned_parquet_snap.promotion]
@@ -185,12 +165,12 @@ PLAN-ROOT SINK
| | tuple-ids=21 row-size=17B cardinality=1.78K cost=180
| | in pipelines: 30(GETNEXT)
| |
-| 33:HASH JOIN [INNER JOIN]
+| 34:HASH JOIN [INNER JOIN]
| | hash predicates: ws_web_site_sk = web_site_sk
| | fk/pk conjuncts: ws_web_site_sk = web_site_sk
-| | runtime filters: RF020[bloom] <- web_site_sk, RF021[min_max] <- web_site_sk
+| | runtime filters: RF018[bloom] <- web_site_sk, RF019[min_max] <- web_site_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=16,17N,20,19 row-size=92B cardinality=212.67M cost=93910750
+| | tuple-ids=16,17N,18,20,19 row-size=100B cardinality=3.61M cost=1596136
| | in pipelines: 25(GETNEXT), 28(OPEN)
| |
| |--28:SCAN HDFS [tpcds_partitioned_parquet_snap.web_site]
@@ -203,12 +183,12 @@ PLAN-ROOT SINK
| | tuple-ids=19 row-size=32B cardinality=66 cost=19
| | in pipelines: 28(GETNEXT)
| |
-| 32:HASH JOIN [INNER JOIN]
+| 33:HASH JOIN [INNER JOIN]
| | hash predicates: ws_item_sk = i_item_sk
| | fk/pk conjuncts: ws_item_sk = i_item_sk
-| | runtime filters: RF022[bloom] <- i_item_sk, RF023[min_max] <- i_item_sk
+| | runtime filters: RF020[bloom] <- i_item_sk, RF021[min_max] <- i_item_sk
| | mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
-| | tuple-ids=16,17N,20 row-size=60B cardinality=215.89M(filtered from 215.89M) cost=94530550
+| | tuple-ids=16,17N,18,20 row-size=68B cardinality=3.67M cost=10116962
| | in pipelines: 25(GETNEXT), 29(OPEN)
| |
| |--29:SCAN HDFS [tpcds_partitioned_parquet_snap.item]
@@ -224,16 +204,37 @@ PLAN-ROOT SINK
| | tuple-ids=20 row-size=8B cardinality=36.00K cost=51588
| | in pipelines: 29(GETNEXT)
| |
+| 32:HASH JOIN [INNER JOIN]
+| | hash predicates: ws_sold_date_sk = d_date_sk
+| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
+| | runtime filters: RF022[bloom] <- d_date_sk
+| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=16,17N,18 row-size=60B cardinality=36.71M cost=16067990
+| | in pipelines: 25(GETNEXT), 27(OPEN)
+| |
+| |--27:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
+| | HDFS partitions=1/1 files=1 size=2.17MB
+| | predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | stored statistics:
+| | table: rows=73.05K size=2.17MB
+| | columns: all
+| | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | tuple-ids=18 row-size=8B cardinality=31 cost=12520
+| | in pipelines: 27(GETNEXT)
+| |
| 31:HASH JOIN [LEFT OUTER JOIN]
| | hash predicates: ws_item_sk = wr_item_sk, ws_order_number = wr_order_number
| | fk/pk conjuncts: ws_item_sk = wr_item_sk, ws_order_number = wr_order_number
| | mem-estimate=10.02GB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| | tuple-ids=16,17N row-size=52B cardinality=215.89M(filtered from 2.16G) cost=662764372
+| | tuple-ids=16,17N row-size=52B cardinality=36.71M(filtered from 2.16G) cost=616804992
| | in pipelines: 25(GETNEXT), 26(OPEN)
| |
| |--26:SCAN HDFS [tpcds_partitioned_parquet_snap.web_returns]
| | HDFS partitions=2114/2114 files=2114 size=20.70MB
-| | runtime filters: RF023[min_max] -> tpcds_partitioned_parquet_snap.web_returns.wr_item_sk, RF022[bloom] -> tpcds_partitioned_parquet_snap.web_returns.wr_item_sk
+| | runtime filters: RF021[min_max] -> tpcds_partitioned_parquet_snap.web_returns.wr_item_sk, RF020[bloom] -> tpcds_partitioned_parquet_snap.web_returns.wr_item_sk
| | stored statistics:
| | table: rows=216.00M size=16.74GB
| | partitions: 2114/2114 rows=216.00M
@@ -245,29 +246,29 @@ PLAN-ROOT SINK
| |
| 25:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales]
| HDFS partitions=1824/1824 files=1824 size=84.18MB
-| runtime filters: RF023[min_max] -> ws_item_sk, RF021[min_max] -> ws_web_site_sk, RF019[min_max] -> ws_promo_sk, RF022[bloom] -> ws_item_sk, RF020[bloom] -> ws_web_site_sk, RF018[bloom] -> ws_promo_sk
+| runtime filters: RF021[min_max] -> ws_item_sk, RF019[min_max] -> ws_web_site_sk, RF017[min_max] -> ws_promo_sk, RF022[bloom] -> ws_sold_date_sk, RF020[bloom] -> ws_item_sk, RF018[bloom] -> ws_web_site_sk, RF016[bloom] -> ws_promo_sk
| stored statistics:
| table: rows=2.16G size=145.75GB
| partitions: 1824/1824 rows=2.16G
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=3.37M
+| extrapolated-rows=disabled max-scan-range-rows=3.37M est-scan-range=32(filtered from 1824)
| mem-estimate=16.00MB mem-reservation=128.00KB thread-reservation=0
-| tuple-ids=16 row-size=32B cardinality=215.89M(filtered from 2.16G) cost=995313239
+| tuple-ids=16 row-size=32B cardinality=36.71M(filtered from 2.16G) cost=16915959
| in pipelines: 25(GETNEXT)
|
|--24:AGGREGATE [FINALIZE]
| | output: sum(cs_ext_sales_price), sum(coalesce(cr_return_amount, CAST(0 AS DECIMAL(7,2)))), sum(cs_net_profit - coalesce(cr_net_loss, CAST(0 AS DECIMAL(7,2))))
| | group by: cp_catalog_page_id
| | mem-estimate=10.00MB mem-reservation=4.75MB spill-buffer=256.00KB thread-reservation=0
-| | tuple-ids=14 row-size=76B cardinality=36.00K cost=184665106
+| | tuple-ids=14 row-size=76B cardinality=36.00K cost=3218768
| | in pipelines: 24(GETNEXT), 13(OPEN)
| |
| 23:HASH JOIN [INNER JOIN]
| | hash predicates: cs_catalog_page_sk = cp_catalog_page_sk
| | fk/pk conjuncts: cs_catalog_page_sk = cp_catalog_page_sk
-| | runtime filters: RF008[bloom] <- cp_catalog_page_sk, RF009[min_max] <- cp_catalog_page_sk
+| | runtime filters: RF009[min_max] <- cp_catalog_page_sk
| | mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
-| | tuple-ids=8,9N,12,13,10,11 row-size=117B cardinality=426.75M cost=186826370
+| | tuple-ids=8,9N,10,12,13,11 row-size=117B cardinality=7.23M cost=3198480
| | in pipelines: 13(GETNEXT), 16(OPEN)
| |
| |--16:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_page]
@@ -281,31 +282,11 @@ PLAN-ROOT SINK
| | in pipelines: 16(GETNEXT)
| |
| 22:HASH JOIN [INNER JOIN]
-| | hash predicates: cs_sold_date_sk = d_date_sk
-| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
-| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=8,9N,12,13,10 row-size=85B cardinality=426.75M cost=186797675
-| | in pipelines: 13(GETNEXT), 15(OPEN)
-| |
-| |--15:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
-| | HDFS partitions=1/1 files=1 size=2.17MB
-| | predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | stored statistics:
-| | table: rows=73.05K size=2.17MB
-| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=73.05K
-| | parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=10 row-size=8B cardinality=7.30K cost=12520
-| | in pipelines: 15(GETNEXT)
-| |
-| 21:HASH JOIN [INNER JOIN]
| | hash predicates: cs_promo_sk = p_promo_sk
| | fk/pk conjuncts: cs_promo_sk = p_promo_sk
-| | runtime filters: RF012[bloom] <- p_promo_sk, RF013[min_max] <- p_promo_sk
+| | runtime filters: RF010[bloom] <- p_promo_sk, RF011[min_max] <- p_promo_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=8,9N,12,13 row-size=77B cardinality=426.75M cost=188084286
+| | tuple-ids=8,9N,10,12,13 row-size=85B cardinality=7.23M cost=3186136
| | in pipelines: 13(GETNEXT), 18(OPEN)
| |
| |--18:SCAN HDFS [tpcds_partitioned_parquet_snap.promotion]
@@ -321,12 +302,12 @@ PLAN-ROOT SINK
| | tuple-ids=13 row-size=17B cardinality=1.78K cost=180
| | in pipelines: 18(GETNEXT)
| |
-| 20:HASH JOIN [INNER JOIN]
+| 21:HASH JOIN [INNER JOIN]
| | hash predicates: cs_item_sk = i_item_sk
| | fk/pk conjuncts: cs_item_sk = i_item_sk
-| | runtime filters: RF014[bloom] <- i_item_sk, RF015[min_max] <- i_item_sk
+| | runtime filters: RF012[bloom] <- i_item_sk, RF013[min_max] <- i_item_sk
| | mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
-| | tuple-ids=8,9N,12 row-size=60B cardinality=431.79M cost=189031315
+| | tuple-ids=8,9N,10,12 row-size=68B cardinality=7.31M cost=20121506
| | in pipelines: 13(GETNEXT), 17(OPEN)
| |
| |--17:SCAN HDFS [tpcds_partitioned_parquet_snap.item]
@@ -342,16 +323,37 @@ PLAN-ROOT SINK
| | tuple-ids=12 row-size=8B cardinality=36.00K cost=51588
| | in pipelines: 17(GETNEXT)
| |
+| 20:HASH JOIN [INNER JOIN]
+| | hash predicates: cs_sold_date_sk = d_date_sk
+| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
+| | runtime filters: RF014[bloom] <- d_date_sk
+| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=8,9N,10 row-size=60B cardinality=73.14M cost=32014144
+| | in pipelines: 13(GETNEXT), 15(OPEN)
+| |
+| |--15:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
+| | HDFS partitions=1/1 files=1 size=2.17MB
+| | predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | stored statistics:
+| | table: rows=73.05K size=2.17MB
+| | columns: all
+| | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | tuple-ids=10 row-size=8B cardinality=31 cost=12520
+| | in pipelines: 15(GETNEXT)
+| |
| 19:HASH JOIN [LEFT OUTER JOIN]
| | hash predicates: cs_item_sk = cr_item_sk, cs_order_number = cr_order_number
| | fk/pk conjuncts: cs_item_sk = cr_item_sk, cs_order_number = cr_order_number
| | mem-estimate=20.05GB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| | tuple-ids=8,9N row-size=52B cardinality=431.79M(filtered from 4.32G) cost=1325569743
+| | tuple-ids=8,9N row-size=52B cardinality=73.14M(filtered from 4.32G) cost=1233575961
| | in pipelines: 13(GETNEXT), 14(OPEN)
| |
| |--14:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_returns]
| | HDFS partitions=2060/2060 files=2060 size=30.47MB
-| | runtime filters: RF015[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk, RF014[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk
+| | runtime filters: RF013[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk, RF012[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk
| | stored statistics:
| | table: rows=432.02M size=32.77GB
| | partitions: 2060/2060 rows=432.02M
@@ -363,29 +365,29 @@ PLAN-ROOT SINK
| |
| 13:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales]
| HDFS partitions=1831/1831 files=1831 size=150.82MB
-| runtime filters: RF015[min_max] -> cs_item_sk, RF013[min_max] -> cs_promo_sk, RF009[min_max] -> cs_catalog_page_sk, RF014[bloom] -> cs_item_sk, RF012[bloom] -> cs_promo_sk, RF008[bloom] -> cs_catalog_page_sk
+| runtime filters: RF013[min_max] -> cs_item_sk, RF011[min_max] -> cs_promo_sk, RF009[min_max] -> cs_catalog_page_sk, RF014[bloom] -> cs_sold_date_sk, RF012[bloom] -> cs_item_sk, RF010[bloom] -> cs_promo_sk
| stored statistics:
| table: rows=4.32G size=280.96GB
| partitions: 1831/1831 rows=4.32G
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=21.52M
+| extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=31(filtered from 1831)
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=8 row-size=32B cardinality=431.79M(filtered from 4.32G) cost=1990691932
+| tuple-ids=8 row-size=32B cardinality=73.14M(filtered from 4.32G) cost=33703686
| in pipelines: 13(GETNEXT)
|
12:AGGREGATE [FINALIZE]
| output: sum(ss_ext_sales_price), sum(coalesce(sr_return_amt, CAST(0 AS DECIMAL(7,2)))), sum(ss_net_profit - coalesce(sr_net_loss, CAST(0 AS DECIMAL(7,2))))
| group by: s_store_id
| mem-estimate=10.00MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=6 row-size=76B cardinality=678 cost=369134831
+| tuple-ids=6 row-size=76B cardinality=678 cost=6275411
| in pipelines: 12(GETNEXT), 01(OPEN)
|
11:HASH JOIN [INNER JOIN]
| hash predicates: ss_store_sk = s_store_sk
| fk/pk conjuncts: ss_store_sk = s_store_sk
-| runtime filters: RF000[bloom] <- s_store_sk, RF001[min_max] <- s_store_sk
+| runtime filters: RF001[min_max] <- s_store_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1N,4,5,2,3 row-size=117B cardinality=853.49M cost=373572545
+| tuple-ids=0,1N,2,4,5,3 row-size=117B cardinality=14.51M cost=6350422
| in pipelines: 01(GETNEXT), 04(OPEN)
|
|--04:SCAN HDFS [tpcds_partitioned_parquet_snap.store]
@@ -399,32 +401,11 @@ PLAN-ROOT SINK
| in pipelines: 04(GETNEXT)
|
10:HASH JOIN [INNER JOIN]
-| hash predicates: ss_sold_date_sk = d_date_sk
-| fk/pk conjuncts: ss_sold_date_sk = d_date_sk
-| runtime filters: RF002[bloom] <- d_date_sk
-| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1N,4,5,2 row-size=85B cardinality=853.49M cost=373578500
-| in pipelines: 01(GETNEXT), 03(OPEN)
-|
-|--03:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
-| HDFS partitions=1/1 files=1 size=2.17MB
-| predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| stored statistics:
-| table: rows=73.05K size=2.17MB
-| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=73.05K
-| parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
-| in pipelines: 03(GETNEXT)
-|
-09:HASH JOIN [INNER JOIN]
| hash predicates: ss_promo_sk = p_promo_sk
| fk/pk conjuncts: ss_promo_sk = p_promo_sk
-| runtime filters: RF004[bloom] <- p_promo_sk, RF005[min_max] <- p_promo_sk
+| runtime filters: RF002[bloom] <- p_promo_sk, RF003[min_max] <- p_promo_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1N,4,5 row-size=77B cardinality=853.49M cost=376157182
+| tuple-ids=0,1N,2,4,5 row-size=85B cardinality=14.51M cost=6394771
| in pipelines: 01(GETNEXT), 06(OPEN)
|
|--06:SCAN HDFS [tpcds_partitioned_parquet_snap.promotion]
@@ -440,12 +421,12 @@ PLAN-ROOT SINK
| tuple-ids=5 row-size=17B cardinality=1.78K cost=180
| in pipelines: 06(GETNEXT)
|
-08:HASH JOIN [INNER JOIN]
+09:HASH JOIN [INNER JOIN]
| hash predicates: ss_item_sk = i_item_sk
| fk/pk conjuncts: ss_item_sk = i_item_sk
-| runtime filters: RF006[bloom] <- i_item_sk, RF007[min_max] <- i_item_sk
+| runtime filters: RF004[bloom] <- i_item_sk, RF005[min_max] <- i_item_sk
| mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
-| tuple-ids=0,1N,4 row-size=60B cardinality=863.56M(filtered from 863.56M) cost=378016972
+| tuple-ids=0,1N,2,4 row-size=68B cardinality=14.68M cost=40360146
| in pipelines: 01(GETNEXT), 05(OPEN)
|
|--05:SCAN HDFS [tpcds_partitioned_parquet_snap.item]
@@ -461,16 +442,37 @@ PLAN-ROOT SINK
| tuple-ids=4 row-size=8B cardinality=36.00K cost=51588
| in pipelines: 05(GETNEXT)
|
+08:HASH JOIN [INNER JOIN]
+| hash predicates: ss_sold_date_sk = d_date_sk
+| fk/pk conjuncts: ss_sold_date_sk = d_date_sk
+| runtime filters: RF006[bloom] <- d_date_sk
+| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,1N,2 row-size=60B cardinality=146.84M cost=64272338
+| in pipelines: 01(GETNEXT), 03(OPEN)
+|
+|--03:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
+| HDFS partitions=1/1 files=1 size=2.17MB
+| predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| stored statistics:
+| table: rows=73.05K size=2.17MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=73.05K
+| parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| tuple-ids=2 row-size=8B cardinality=31 cost=12520
+| in pipelines: 03(GETNEXT)
+|
07:HASH JOIN [LEFT OUTER JOIN]
| hash predicates: ss_item_sk = sr_item_sk, ss_ticket_number = sr_ticket_number
| fk/pk conjuncts: ss_item_sk = sr_item_sk, ss_ticket_number = sr_ticket_number
| mem-estimate=40.09GB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,1N row-size=52B cardinality=863.56M(filtered from 8.64G) cost=2651048474
+| tuple-ids=0,1N row-size=52B cardinality=146.84M(filtered from 8.64G) cost=2467209606
| in pipelines: 01(GETNEXT), 02(OPEN)
|
|--02:SCAN HDFS [tpcds_partitioned_parquet_snap.store_returns]
| HDFS partitions=2004/2004 files=2004 size=33.63MB
-| runtime filters: RF007[min_max] -> tpcds_partitioned_parquet_snap.store_returns.sr_item_sk, RF006[bloom] -> tpcds_partitioned_parquet_snap.store_returns.sr_item_sk
+| runtime filters: RF005[min_max] -> tpcds_partitioned_parquet_snap.store_returns.sr_item_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.store_returns.sr_item_sk
| stored statistics:
| table: rows=863.99M size=48.14GB
| partitions: 2004/2004 rows=863.99M
@@ -482,21 +484,21 @@ PLAN-ROOT SINK
|
01:SCAN HDFS [tpcds_partitioned_parquet_snap.store_sales]
HDFS partitions=1824/1824 files=1824 size=199.33MB
- runtime filters: RF007[min_max] -> ss_item_sk, RF005[min_max] -> ss_promo_sk, RF001[min_max] -> ss_store_sk, RF006[bloom] -> ss_item_sk, RF004[bloom] -> ss_promo_sk, RF000[bloom] -> ss_store_sk, RF002[bloom] -> ss_sold_date_sk
+ runtime filters: RF005[min_max] -> ss_item_sk, RF003[min_max] -> ss_promo_sk, RF001[min_max] -> ss_store_sk, RF006[bloom] -> ss_sold_date_sk, RF004[bloom] -> ss_item_sk, RF002[bloom] -> ss_promo_sk
stored statistics:
table: rows=8.64G size=389.90GB
partitions: 1824/1824 rows=8.64G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=390.22M
+ extrapolated-rows=disabled max-scan-range-rows=390.22M est-scan-range=32(filtered from 1824)
mem-estimate=16.00MB mem-reservation=8.00MB thread-reservation=0
- tuple-ids=0 row-size=32B cardinality=863.56M(filtered from 8.64G) cost=3981282136
+ tuple-ids=0 row-size=32B cardinality=146.84M(filtered from 8.64G) cost=67664334
in pipelines: 01(GETNEXT)
---- DISTRIBUTEDPLAN
-Max Per-Host Resource Reservation: Memory=1.78GB Threads=169
-Per-Host Resource Estimates: Memory=12.56GB
+Max Per-Host Resource Reservation: Memory=1.34GB Threads=126
+Per-Host Resource Estimates: Memory=10.89GB
F26:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=4.07MB mem-reservation=4.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[539] cpu-comparison-result=774 [max(1 (self) vs 774 (sum children))]
+| max-parallelism=1 segment-costs=[539] cpu-comparison-result=671 [max(1 (self) vs 671 (sum children))]
PLAN-ROOT SINK
| output exprs: CASE valid_tid(26,27,28) WHEN 26 THEN channel WHEN 27 THEN channel WHEN 28 THEN NULL END, CASE valid_tid(26,27,28) WHEN 26 THEN id WHEN 27 THEN NULL WHEN 28 THEN NULL END, aggif(valid_tid(26,27,28) IN (26, 27, 28), CASE valid_tid(26,27,28) WHEN 26 THEN sum(sales) WHEN 27 THEN sum(sales) WHEN 28 THEN sum(sales) END), aggif(valid_tid(26,27,28) IN (26, 27, 28), CASE valid_tid(26,27,28) WHEN 26 THEN sum(`returns`) WHEN 27 THEN sum(`returns`) WHEN 28 THEN sum(`returns`) END), [...]
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=500
@@ -510,7 +512,7 @@ PLAN-ROOT SINK
|
F25:PLAN FRAGMENT [HASH(CASE valid_tid(26,27,28) WHEN 26 THEN murmur_hash(channel) WHEN 27 THEN murmur_hash(channel) WHEN 28 THEN murmur_hash(NULL) END,CASE valid_tid(26,27,28) WHEN 26 THEN murmur_hash(id) WHEN 27 THEN murmur_hash(NULL) WHEN 28 THEN murmur_hash(NULL) END)] hosts=10 instances=10 (adjusted from 120)
Per-Instance Resources: mem-estimate=57.48MB mem-reservation=7.75MB thread-reservation=1
-max-parallelism=10 segment-costs=[265114, 221854, 313895, 220] cpu-comparison-result=774 [max(10 (self) vs 774 (sum children))]
+max-parallelism=10 segment-costs=[265114, 221854, 313895, 220] cpu-comparison-result=671 [max(10 (self) vs 671 (sum children))]
39:TOP-N [LIMIT=100]
| order by: CASE valid_tid(26,27,28) WHEN 26 THEN channel WHEN 27 THEN channel WHEN 28 THEN NULL END ASC, CASE valid_tid(26,27,28) WHEN 26 THEN id WHEN 27 THEN NULL WHEN 28 THEN NULL END ASC
| mem-estimate=7.03KB mem-reservation=0B thread-reservation=0
@@ -544,8 +546,8 @@ max-parallelism=10 segment-costs=[265114, 221854, 313895, 220] cpu-comparison-re
| in pipelines: 47(GETNEXT), 55(GETNEXT), 63(GETNEXT)
|
F24:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=58.54MB mem-reservation=7.94MB thread-reservation=1
-max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-comparison-result=774 [max(120 (self) vs 774 (sum children))]
+Per-Instance Resources: mem-estimate=56.20MB mem-reservation=7.94MB thread-reservation=1
+max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-comparison-result=671 [max(120 (self) vs 671 (sum children))]
37:AGGREGATE [STREAMING]
| Class 0
| output: sum(sales), sum(returns), sum(profit)
@@ -573,80 +575,43 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | in pipelines: 63(GETNEXT), 25(OPEN)
| |
| 62:EXCHANGE [HASH(web_site_id)]
-| | mem-estimate=258.06KB mem-reservation=0B thread-reservation=0
+| | mem-estimate=129.15KB mem-reservation=0B thread-reservation=0
| | tuple-ids=22 row-size=76B cardinality=33 cost=15
| | in pipelines: 25(GETNEXT)
| |
-| F18:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=100 (adjusted from 120)
-| Per-Instance Resources: mem-estimate=61.72MB mem-reservation=2.00MB thread-reservation=1
-| max-parallelism=100 segment-costs=[966479381, 178] cpu-comparison-result=220 [max(220 (self) vs 197 (sum children))]
+| F18:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=50 (adjusted from 120)
+| Per-Instance Resources: mem-estimate=58.20MB mem-reservation=2.00MB thread-reservation=1
+| max-parallelism=50 segment-costs=[441159182, 178] cpu-comparison-result=147 [max(70 (self) vs 147 (sum children))]
| 36:AGGREGATE [STREAMING]
| | output: sum(ws_ext_sales_price), sum(coalesce(wr_return_amt, CAST(0 AS DECIMAL(7,2)))), sum(ws_net_profit - coalesce(wr_net_loss, CAST(0 AS DECIMAL(7,2))))
| | group by: web_site_id
| | mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=22 row-size=76B cardinality=33 cost=90915562
+| | tuple-ids=22 row-size=76B cardinality=33 cost=1555315
| | in pipelines: 25(GETNEXT)
| |
| 35:HASH JOIN [INNER JOIN, BROADCAST]
| | hash-table-id=10
-| | hash predicates: ws_sold_date_sk = d_date_sk
-| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=16,17N,20,19,21,18 row-size=117B cardinality=210.19M cost=91998202
-| | in pipelines: 25(GETNEXT), 27(OPEN)
-| |
-| |--F37:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=19.44MB mem-reservation=19.38MB thread-reservation=1
-| | | max-parallelism=10 segment-costs=[17005]
-| | JOIN BUILD
-| | | join-table-id=10 plan-id=11 cohort-id=03
-| | | build expressions: d_date_sk
-| | | mem-estimate=19.38MB mem-reservation=19.38MB spill-buffer=64.00KB thread-reservation=0 cost=7305
-| | |
-| | 61:EXCHANGE [BROADCAST]
-| | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=18 row-size=8B cardinality=7.30K cost=9700
-| | | in pipelines: 27(GETNEXT)
-| | |
-| | F22:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[12835]
-| | 27:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| | HDFS partitions=1/1 files=1 size=2.17MB
-| | predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | stored statistics:
-| | table: rows=73.05K size=2.17MB
-| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=73.05K
-| | parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=18 row-size=8B cardinality=7.30K cost=12520
-| | in pipelines: 27(GETNEXT)
-| |
-| 34:HASH JOIN [INNER JOIN, BROADCAST]
-| | hash-table-id=11
| | hash predicates: ws_promo_sk = p_promo_sk
| | fk/pk conjuncts: ws_promo_sk = p_promo_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=16,17N,20,19,21 row-size=109B cardinality=210.19M cost=92634607
+| | tuple-ids=16,17N,18,20,19,21 row-size=117B cardinality=3.57M cost=1574382
| | in pipelines: 25(GETNEXT), 30(OPEN)
| |
-| |--F38:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=20.42MB mem-reservation=20.38MB thread-reservation=1 runtime-filters-memory=1.00MB
+| |--F37:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=10
+| | | Per-Instance Resources: mem-estimate=10.74MB mem-reservation=10.69MB thread-reservation=1 runtime-filters-memory=1.00MB
| | | max-parallelism=10 segment-costs=[4139]
| | JOIN BUILD
-| | | join-table-id=11 plan-id=12 cohort-id=03
+| | | join-table-id=10 plan-id=11 cohort-id=03
| | | build expressions: p_promo_sk
-| | | runtime filters: RF018[bloom] <- p_promo_sk, RF019[min_max] <- p_promo_sk
-| | | mem-estimate=19.38MB mem-reservation=19.38MB spill-buffer=64.00KB thread-reservation=0 cost=1779
+| | | runtime filters: RF016[bloom] <- p_promo_sk, RF017[min_max] <- p_promo_sk
+| | | mem-estimate=9.69MB mem-reservation=9.69MB spill-buffer=64.00KB thread-reservation=0 cost=1779
| | |
-| | 60:EXCHANGE [BROADCAST]
+| | 61:EXCHANGE [BROADCAST]
| | | mem-estimate=50.53KB mem-reservation=0B thread-reservation=0
| | | tuple-ids=21 row-size=17B cardinality=1.78K cost=2360
| | | in pipelines: 30(GETNEXT)
| | |
-| | F21:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | F22:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| | Per-Instance Resources: mem-estimate=16.08MB mem-reservation=32.00KB thread-reservation=1
| | max-parallelism=1 segment-costs=[300]
| | 30:SCAN HDFS [tpcds_partitioned_parquet_snap.promotion, RANDOM]
@@ -662,29 +627,29 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | tuple-ids=21 row-size=17B cardinality=1.78K cost=180
| | in pipelines: 30(GETNEXT)
| |
-| 33:HASH JOIN [INNER JOIN, BROADCAST]
-| | hash-table-id=12
+| 34:HASH JOIN [INNER JOIN, BROADCAST]
+| | hash-table-id=11
| | hash predicates: ws_web_site_sk = web_site_sk
| | fk/pk conjuncts: ws_web_site_sk = web_site_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=16,17N,20,19 row-size=92B cardinality=212.67M cost=93910684
+| | tuple-ids=16,17N,18,20,19 row-size=100B cardinality=3.61M cost=1596070
| | in pipelines: 25(GETNEXT), 28(OPEN)
| |
-| |--F39:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=20.39MB mem-reservation=20.38MB thread-reservation=1 runtime-filters-memory=1.00MB
+| |--F38:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=10
+| | | Per-Instance Resources: mem-estimate=10.70MB mem-reservation=10.69MB thread-reservation=1 runtime-filters-memory=1.00MB
| | | max-parallelism=10 segment-costs=[146]
| | JOIN BUILD
-| | | join-table-id=12 plan-id=13 cohort-id=03
+| | | join-table-id=11 plan-id=12 cohort-id=03
| | | build expressions: web_site_sk
-| | | runtime filters: RF020[bloom] <- web_site_sk, RF021[min_max] <- web_site_sk
-| | | mem-estimate=19.38MB mem-reservation=19.38MB spill-buffer=64.00KB thread-reservation=0 cost=66
+| | | runtime filters: RF018[bloom] <- web_site_sk, RF019[min_max] <- web_site_sk
+| | | mem-estimate=9.69MB mem-reservation=9.69MB spill-buffer=64.00KB thread-reservation=0 cost=66
| | |
-| | 59:EXCHANGE [BROADCAST]
+| | 60:EXCHANGE [BROADCAST]
| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
| | | tuple-ids=19 row-size=32B cardinality=66 cost=80
| | | in pipelines: 28(GETNEXT)
| | |
-| | F20:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | F21:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| | Per-Instance Resources: mem-estimate=16.14MB mem-reservation=16.00KB thread-reservation=1
| | max-parallelism=1 segment-costs=[26]
| | 28:SCAN HDFS [tpcds_partitioned_parquet_snap.web_site, RANDOM]
@@ -697,29 +662,29 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | tuple-ids=19 row-size=32B cardinality=66 cost=19
| | in pipelines: 28(GETNEXT)
| |
-| 32:HASH JOIN [INNER JOIN, BROADCAST]
-| | hash-table-id=13
+| 33:HASH JOIN [INNER JOIN, BROADCAST]
+| | hash-table-id=12
| | hash predicates: ws_item_sk = i_item_sk
| | fk/pk conjuncts: ws_item_sk = i_item_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
-| | tuple-ids=16,17N,20 row-size=60B cardinality=215.89M(filtered from 215.89M) cost=94494550
+| | tuple-ids=16,17N,18,20 row-size=68B cardinality=3.67M cost=10080962
| | in pipelines: 25(GETNEXT), 29(OPEN)
| |
-| |--F40:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=30.07MB mem-reservation=29.75MB thread-reservation=1 runtime-filters-memory=1.00MB
+| |--F39:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=10
+| | | Per-Instance Resources: mem-estimate=15.70MB mem-reservation=15.38MB thread-reservation=1 runtime-filters-memory=1.00MB
| | | max-parallelism=10 segment-costs=[83840]
| | JOIN BUILD
-| | | join-table-id=13 plan-id=14 cohort-id=03
+| | | join-table-id=12 plan-id=13 cohort-id=03
| | | build expressions: i_item_sk
-| | | runtime filters: RF022[bloom] <- i_item_sk, RF023[min_max] <- i_item_sk
-| | | mem-estimate=28.75MB mem-reservation=28.75MB spill-buffer=128.00KB thread-reservation=0 cost=36000
+| | | runtime filters: RF020[bloom] <- i_item_sk, RF021[min_max] <- i_item_sk
+| | | mem-estimate=14.38MB mem-reservation=14.38MB spill-buffer=128.00KB thread-reservation=0 cost=36000
| | |
-| | 58:EXCHANGE [BROADCAST]
+| | 59:EXCHANGE [BROADCAST]
| | | mem-estimate=329.25KB mem-reservation=0B thread-reservation=0
| | | tuple-ids=20 row-size=8B cardinality=36.00K cost=47840
| | | in pipelines: 29(GETNEXT)
| | |
-| | F19:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
+| | F20:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=512.00KB thread-reservation=1
| | max-parallelism=4 segment-costs=[53143]
| | 29:SCAN HDFS [tpcds_partitioned_parquet_snap.item, RANDOM]
@@ -735,21 +700,59 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | tuple-ids=20 row-size=8B cardinality=36.00K cost=51588
| | in pipelines: 29(GETNEXT)
| |
+| 32:HASH JOIN [INNER JOIN, BROADCAST]
+| | hash-table-id=13
+| | hash predicates: ws_sold_date_sk = d_date_sk
+| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
+| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=16,17N,18 row-size=60B cardinality=36.71M cost=16067959
+| | in pipelines: 25(GETNEXT), 27(OPEN)
+| |
+| |--F40:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=10
+| | | Per-Instance Resources: mem-estimate=10.70MB mem-reservation=10.69MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[71]
+| | JOIN BUILD
+| | | join-table-id=13 plan-id=14 cohort-id=03
+| | | build expressions: d_date_sk
+| | | runtime filters: RF022[bloom] <- d_date_sk
+| | | mem-estimate=9.69MB mem-reservation=9.69MB spill-buffer=64.00KB thread-reservation=0 cost=31
+| | |
+| | 58:EXCHANGE [BROADCAST]
+| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=18 row-size=8B cardinality=31 cost=40
+| | | in pipelines: 27(GETNEXT)
+| | |
+| | F19:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| | max-parallelism=1 segment-costs=[12521]
+| | 27:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| | HDFS partitions=1/1 files=1 size=2.17MB
+| | predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | stored statistics:
+| | table: rows=73.05K size=2.17MB
+| | columns: all
+| | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | tuple-ids=18 row-size=8B cardinality=31 cost=12520
+| | in pipelines: 27(GETNEXT)
+| |
| 31:HASH JOIN [LEFT OUTER JOIN, PARTITIONED]
| | hash-table-id=14
| | hash predicates: ws_item_sk = wr_item_sk, ws_order_number = wr_order_number
| | fk/pk conjuncts: ws_item_sk = wr_item_sk, ws_order_number = wr_order_number
| | mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| | tuple-ids=16,17N row-size=52B cardinality=215.89M(filtered from 2.16G) cost=446761686
+| | tuple-ids=16,17N row-size=52B cardinality=36.71M(filtered from 2.16G) cost=400802306
| | in pipelines: 25(GETNEXT), 26(OPEN)
| |
-| |--F41:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=100 (adjusted from 120)
-| | | Per-Instance Resources: mem-estimate=113.81MB mem-reservation=34.00MB thread-reservation=1
-| | | max-parallelism=100 segment-costs=[259872831]
+| |--F41:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=50 (adjusted from 120)
+| | | Per-Instance Resources: mem-estimate=216.45MB mem-reservation=34.00MB thread-reservation=1
+| | | max-parallelism=50 segment-costs=[259872831]
| | JOIN BUILD
| | | join-table-id=14 plan-id=15 cohort-id=03
| | | build expressions: wr_item_sk, wr_order_number
-| | | mem-estimate=102.64MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 cost=216002686
+| | | mem-estimate=205.28MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 cost=216002686
| | |
| | 57:EXCHANGE [HASH(wr_item_sk,wr_order_number)]
| | | mem-estimate=11.17MB mem-reservation=0B thread-reservation=0
@@ -758,11 +761,11 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | |
| | F17:PLAN FRAGMENT [RANDOM] hosts=10 instances=50 (adjusted from 120)
| | Per-Host Shared Resources: mem-estimate=1.00MB mem-reservation=1.00MB thread-reservation=0 runtime-filters-memory=1.00MB
-| | Per-Instance Resources: mem-estimate=25.38MB mem-reservation=256.00KB thread-reservation=1
+| | Per-Instance Resources: mem-estimate=20.69MB mem-reservation=256.00KB thread-reservation=1
| | max-parallelism=50 segment-costs=[451704816]
| | 26:SCAN HDFS [tpcds_partitioned_parquet_snap.web_returns, RANDOM]
| | HDFS partitions=2114/2114 files=2114 size=20.70MB
-| | runtime filters: RF023[min_max] -> tpcds_partitioned_parquet_snap.web_returns.wr_item_sk, RF022[bloom] -> tpcds_partitioned_parquet_snap.web_returns.wr_item_sk
+| | runtime filters: RF021[min_max] -> tpcds_partitioned_parquet_snap.web_returns.wr_item_sk, RF020[bloom] -> tpcds_partitioned_parquet_snap.web_returns.wr_item_sk
| | stored statistics:
| | table: rows=216.00M size=16.74GB
| | partitions: 2114/2114 rows=216.00M
@@ -773,24 +776,24 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | in pipelines: 26(GETNEXT)
| |
| 56:EXCHANGE [HASH(ws_item_sk,ws_order_number)]
-| | mem-estimate=14.22MB mem-reservation=0B thread-reservation=0
-| | tuple-ids=16 row-size=32B cardinality=215.89M(filtered from 2.16G) cost=55764090
+| | mem-estimate=10.70MB mem-reservation=0B thread-reservation=0
+| | tuple-ids=16 row-size=32B cardinality=36.71M(filtered from 2.16G) cost=9482188
| | in pipelines: 25(GETNEXT)
| |
-| F16:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
-| Per-Host Shared Resources: mem-estimate=3.00MB mem-reservation=3.00MB thread-reservation=0 runtime-filters-memory=3.00MB
-| Per-Instance Resources: mem-estimate=30.06MB mem-reservation=128.00KB thread-reservation=1
-| max-parallelism=160 segment-costs=[1551442921]
+| F16:PLAN FRAGMENT [RANDOM] hosts=10 instances=20 (adjusted from 120)
+| Per-Host Shared Resources: mem-estimate=4.00MB mem-reservation=4.00MB thread-reservation=0 runtime-filters-memory=4.00MB
+| Per-Instance Resources: mem-estimate=23.03MB mem-reservation=128.00KB thread-reservation=1
+| max-parallelism=20 segment-costs=[111480872]
| 25:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales, RANDOM]
| HDFS partitions=1824/1824 files=1824 size=84.18MB
-| runtime filters: RF023[min_max] -> ws_item_sk, RF021[min_max] -> ws_web_site_sk, RF019[min_max] -> ws_promo_sk, RF022[bloom] -> ws_item_sk, RF020[bloom] -> ws_web_site_sk, RF018[bloom] -> ws_promo_sk
+| runtime filters: RF021[min_max] -> ws_item_sk, RF019[min_max] -> ws_web_site_sk, RF017[min_max] -> ws_promo_sk, RF022[bloom] -> ws_sold_date_sk, RF020[bloom] -> ws_item_sk, RF018[bloom] -> ws_web_site_sk, RF016[bloom] -> ws_promo_sk
| stored statistics:
| table: rows=2.16G size=145.75GB
| partitions: 1824/1824 rows=2.16G
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=3.37M
+| extrapolated-rows=disabled max-scan-range-rows=3.37M est-scan-range=32(filtered from 1824)
| mem-estimate=16.00MB mem-reservation=128.00KB thread-reservation=0
-| tuple-ids=16 row-size=32B cardinality=215.89M(filtered from 2.16G) cost=995313239
+| tuple-ids=16 row-size=32B cardinality=36.71M(filtered from 2.16G) cost=16915959
| in pipelines: 25(GETNEXT)
|
|--55:AGGREGATE [FINALIZE]
@@ -801,18 +804,18 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | in pipelines: 55(GETNEXT), 13(OPEN)
| |
| 54:EXCHANGE [HASH(cp_catalog_page_id)]
-| | mem-estimate=9.64MB mem-reservation=0B thread-reservation=0
+| | mem-estimate=7.29MB mem-reservation=0B thread-reservation=0
| | tuple-ids=14 row-size=76B cardinality=36.00K cost=16585
| | in pipelines: 13(GETNEXT)
| |
-| F10:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=120
-| Per-Instance Resources: mem-estimate=61.72MB mem-reservation=5.00MB thread-reservation=1
-| max-parallelism=200 segment-costs=[1951577513, 194745] cpu-comparison-result=267 [max(240 (self) vs 267 (sum children))]
+| F10:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=90 (adjusted from 120)
+| Per-Instance Resources: mem-estimate=58.55MB mem-reservation=5.00MB thread-reservation=1
+| max-parallelism=90 segment-costs=[891170974, 194745] cpu-comparison-result=237 [max(120 (self) vs 237 (sum children))]
| 24:AGGREGATE [STREAMING]
| | output: sum(cs_ext_sales_price), sum(coalesce(cr_return_amount, CAST(0 AS DECIMAL(7,2)))), sum(cs_net_profit - coalesce(cr_net_loss, CAST(0 AS DECIMAL(7,2))))
| | group by: cp_catalog_page_id
| | mem-estimate=10.00MB mem-reservation=5.00MB spill-buffer=256.00KB thread-reservation=0
-| | tuple-ids=14 row-size=76B cardinality=36.00K cost=195834351
+| | tuple-ids=14 row-size=76B cardinality=36.00K cost=12273052
| | in pipelines: 13(GETNEXT)
| |
| 23:HASH JOIN [INNER JOIN, BROADCAST]
@@ -820,17 +823,17 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | hash predicates: cs_catalog_page_sk = cp_catalog_page_sk
| | fk/pk conjuncts: cs_catalog_page_sk = cp_catalog_page_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
-| | tuple-ids=8,9N,12,13,10,11 row-size=117B cardinality=426.75M cost=186790370
+| | tuple-ids=8,9N,10,12,13,11 row-size=117B cardinality=7.23M cost=3162480
| | in pipelines: 13(GETNEXT), 16(OPEN)
| |
| |--F32:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=36.63MB mem-reservation=35.50MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | Per-Instance Resources: mem-estimate=27.01MB mem-reservation=25.88MB thread-reservation=1
| | | max-parallelism=10 segment-costs=[83840]
| | JOIN BUILD
| | | join-table-id=05 plan-id=06 cohort-id=02
| | | build expressions: cp_catalog_page_sk
-| | | runtime filters: RF008[bloom] <- cp_catalog_page_sk, RF009[min_max] <- cp_catalog_page_sk
-| | | mem-estimate=34.50MB mem-reservation=34.50MB spill-buffer=128.00KB thread-reservation=0 cost=36000
+| | | runtime filters: RF009[min_max] <- cp_catalog_page_sk
+| | | mem-estimate=25.88MB mem-reservation=25.88MB spill-buffer=128.00KB thread-reservation=0 cost=36000
| | |
| | 53:EXCHANGE [BROADCAST]
| | | mem-estimate=1.13MB mem-reservation=0B thread-reservation=0
@@ -852,64 +855,27 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| |
| 22:HASH JOIN [INNER JOIN, BROADCAST]
| | hash-table-id=06
-| | hash predicates: cs_sold_date_sk = d_date_sk
-| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
-| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=8,9N,12,13,10 row-size=85B cardinality=426.75M cost=186790370
-| | in pipelines: 13(GETNEXT), 15(OPEN)
-| |
-| |--F33:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=23.32MB mem-reservation=23.25MB thread-reservation=1
-| | | max-parallelism=10 segment-costs=[17005]
-| | JOIN BUILD
-| | | join-table-id=06 plan-id=07 cohort-id=02
-| | | build expressions: d_date_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
-| | |
-| | 52:EXCHANGE [BROADCAST]
-| | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=10 row-size=8B cardinality=7.30K cost=9700
-| | | in pipelines: 15(GETNEXT)
-| | |
-| | F13:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[12835]
-| | 15:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| | HDFS partitions=1/1 files=1 size=2.17MB
-| | predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | stored statistics:
-| | table: rows=73.05K size=2.17MB
-| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=73.05K
-| | parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=10 row-size=8B cardinality=7.30K cost=12520
-| | in pipelines: 15(GETNEXT)
-| |
-| 21:HASH JOIN [INNER JOIN, BROADCAST]
-| | hash-table-id=07
| | hash predicates: cs_promo_sk = p_promo_sk
| | fk/pk conjuncts: cs_promo_sk = p_promo_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=8,9N,12,13 row-size=77B cardinality=426.75M cost=188082507
+| | tuple-ids=8,9N,10,12,13 row-size=85B cardinality=7.23M cost=3184357
| | in pipelines: 13(GETNEXT), 18(OPEN)
| |
-| |--F34:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.30MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| |--F33:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
+| | | Per-Instance Resources: mem-estimate=18.49MB mem-reservation=18.44MB thread-reservation=1 runtime-filters-memory=1.00MB
| | | max-parallelism=10 segment-costs=[4139]
| | JOIN BUILD
-| | | join-table-id=07 plan-id=08 cohort-id=02
+| | | join-table-id=06 plan-id=07 cohort-id=02
| | | build expressions: p_promo_sk
-| | | runtime filters: RF012[bloom] <- p_promo_sk, RF013[min_max] <- p_promo_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=1779
+| | | runtime filters: RF010[bloom] <- p_promo_sk, RF011[min_max] <- p_promo_sk
+| | | mem-estimate=17.44MB mem-reservation=17.44MB spill-buffer=64.00KB thread-reservation=0 cost=1779
| | |
-| | 51:EXCHANGE [BROADCAST]
+| | 52:EXCHANGE [BROADCAST]
| | | mem-estimate=50.53KB mem-reservation=0B thread-reservation=0
| | | tuple-ids=13 row-size=17B cardinality=1.78K cost=2360
| | | in pipelines: 18(GETNEXT)
| | |
-| | F12:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | F13:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| | Per-Instance Resources: mem-estimate=16.08MB mem-reservation=32.00KB thread-reservation=1
| | max-parallelism=1 segment-costs=[300]
| | 18:SCAN HDFS [tpcds_partitioned_parquet_snap.promotion, RANDOM]
@@ -925,29 +891,29 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | tuple-ids=13 row-size=17B cardinality=1.78K cost=180
| | in pipelines: 18(GETNEXT)
| |
-| 20:HASH JOIN [INNER JOIN, BROADCAST]
-| | hash-table-id=08
+| 21:HASH JOIN [INNER JOIN, BROADCAST]
+| | hash-table-id=07
| | hash predicates: cs_item_sk = i_item_sk
| | fk/pk conjuncts: cs_item_sk = i_item_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
-| | tuple-ids=8,9N,12 row-size=60B cardinality=431.79M cost=188995315
+| | tuple-ids=8,9N,10,12 row-size=68B cardinality=7.31M cost=20085506
| | in pipelines: 13(GETNEXT), 17(OPEN)
| |
-| |--F35:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=35.82MB mem-reservation=35.50MB thread-reservation=1 runtime-filters-memory=1.00MB
+| |--F34:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
+| | | Per-Instance Resources: mem-estimate=27.20MB mem-reservation=26.88MB thread-reservation=1 runtime-filters-memory=1.00MB
| | | max-parallelism=10 segment-costs=[83840]
| | JOIN BUILD
-| | | join-table-id=08 plan-id=09 cohort-id=02
+| | | join-table-id=07 plan-id=08 cohort-id=02
| | | build expressions: i_item_sk
-| | | runtime filters: RF014[bloom] <- i_item_sk, RF015[min_max] <- i_item_sk
-| | | mem-estimate=34.50MB mem-reservation=34.50MB spill-buffer=128.00KB thread-reservation=0 cost=36000
+| | | runtime filters: RF012[bloom] <- i_item_sk, RF013[min_max] <- i_item_sk
+| | | mem-estimate=25.88MB mem-reservation=25.88MB spill-buffer=128.00KB thread-reservation=0 cost=36000
| | |
-| | 50:EXCHANGE [BROADCAST]
+| | 51:EXCHANGE [BROADCAST]
| | | mem-estimate=329.25KB mem-reservation=0B thread-reservation=0
| | | tuple-ids=12 row-size=8B cardinality=36.00K cost=47840
| | | in pipelines: 17(GETNEXT)
| | |
-| | F11:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
+| | F12:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=512.00KB thread-reservation=1
| | max-parallelism=4 segment-costs=[53143]
| | 17:SCAN HDFS [tpcds_partitioned_parquet_snap.item, RANDOM]
@@ -963,21 +929,59 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | tuple-ids=12 row-size=8B cardinality=36.00K cost=51588
| | in pipelines: 17(GETNEXT)
| |
-| 19:HASH JOIN [LEFT OUTER JOIN, PARTITIONED]
-| | hash-table-id=09
-| | hash predicates: cs_item_sk = cr_item_sk, cs_order_number = cr_order_number
-| | fk/pk conjuncts: cs_item_sk = cr_item_sk, cs_order_number = cr_order_number
-| | mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| | tuple-ids=8,9N row-size=52B cardinality=431.79M(filtered from 4.32G) cost=893552752
-| | in pipelines: 13(GETNEXT), 14(OPEN)
+| 20:HASH JOIN [INNER JOIN, BROADCAST]
+| | hash-table-id=08
+| | hash predicates: cs_sold_date_sk = d_date_sk
+| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
+| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=8,9N,10 row-size=60B cardinality=73.14M cost=32014113
+| | in pipelines: 13(GETNEXT), 15(OPEN)
+| |
+| |--F35:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
+| | | Per-Instance Resources: mem-estimate=18.45MB mem-reservation=18.44MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[71]
+| | JOIN BUILD
+| | | join-table-id=08 plan-id=09 cohort-id=02
+| | | build expressions: d_date_sk
+| | | runtime filters: RF014[bloom] <- d_date_sk
+| | | mem-estimate=17.44MB mem-reservation=17.44MB spill-buffer=64.00KB thread-reservation=0 cost=31
+| | |
+| | 50:EXCHANGE [BROADCAST]
+| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=10 row-size=8B cardinality=31 cost=40
+| | | in pipelines: 15(GETNEXT)
+| | |
+| | F11:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| | max-parallelism=1 segment-costs=[12521]
+| | 15:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| | HDFS partitions=1/1 files=1 size=2.17MB
+| | predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | stored statistics:
+| | table: rows=73.05K size=2.17MB
+| | columns: all
+| | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | tuple-ids=10 row-size=8B cardinality=31 cost=12520
+| | in pipelines: 15(GETNEXT)
+| |
+| 19:HASH JOIN [LEFT OUTER JOIN, PARTITIONED]
+| | hash-table-id=09
+| | hash predicates: cs_item_sk = cr_item_sk, cs_order_number = cr_order_number
+| | fk/pk conjuncts: cs_item_sk = cr_item_sk, cs_order_number = cr_order_number
+| | mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
+| | tuple-ids=8,9N row-size=52B cardinality=73.14M(filtered from 4.32G) cost=801558970
+| | in pipelines: 13(GETNEXT), 14(OPEN)
| |
-| |--F36:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=120
-| | | Per-Instance Resources: mem-estimate=183.41MB mem-reservation=34.00MB thread-reservation=1
-| | | max-parallelism=200 segment-costs=[519759641]
+| |--F36:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=90 (adjusted from 120)
+| | | Per-Instance Resources: mem-estimate=240.43MB mem-reservation=34.00MB thread-reservation=1
+| | | max-parallelism=90 segment-costs=[519759641]
| | JOIN BUILD
| | | join-table-id=09 plan-id=10 cohort-id=02
| | | build expressions: cr_item_sk, cr_order_number
-| | | mem-estimate=171.07MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 cost=432016991
+| | | mem-estimate=228.09MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 cost=432016991
| | |
| | 49:EXCHANGE [HASH(cr_item_sk,cr_order_number)]
| | | mem-estimate=12.34MB mem-reservation=0B thread-reservation=0
@@ -986,11 +990,11 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | |
| | F09:PLAN FRAGMENT [RANDOM] hosts=10 instances=100 (adjusted from 120)
| | Per-Host Shared Resources: mem-estimate=1.00MB mem-reservation=1.00MB thread-reservation=0 runtime-filters-memory=1.00MB
-| | Per-Instance Resources: mem-estimate=27.25MB mem-reservation=32.00KB thread-reservation=1
+| | Per-Instance Resources: mem-estimate=24.44MB mem-reservation=32.00KB thread-reservation=1
| | max-parallelism=100 segment-costs=[903433931]
| | 14:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_returns, RANDOM]
| | HDFS partitions=2060/2060 files=2060 size=30.47MB
-| | runtime filters: RF015[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk, RF014[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk
+| | runtime filters: RF013[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk, RF012[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk
| | stored statistics:
| | table: rows=432.02M size=32.77GB
| | partitions: 2060/2060 rows=432.02M
@@ -1001,24 +1005,24 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | in pipelines: 14(GETNEXT)
| |
| 48:EXCHANGE [HASH(cs_item_sk,cs_order_number)]
-| | mem-estimate=14.22MB mem-reservation=0B thread-reservation=0
-| | tuple-ids=8 row-size=32B cardinality=431.79M(filtered from 4.32G) cost=111531848
+| | mem-estimate=11.05MB mem-reservation=0B thread-reservation=0
+| | tuple-ids=8 row-size=32B cardinality=73.14M(filtered from 4.32G) cost=18892496
| | in pipelines: 13(GETNEXT)
| |
-| F08:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+| F08:PLAN FRAGMENT [RANDOM] hosts=10 instances=30 (adjusted from 120)
| Per-Host Shared Resources: mem-estimate=3.00MB mem-reservation=3.00MB thread-reservation=0 runtime-filters-memory=3.00MB
-| Per-Instance Resources: mem-estimate=32.88MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=320 segment-costs=[3102987871]
+| Per-Instance Resources: mem-estimate=28.66MB mem-reservation=1.00MB thread-reservation=1
+| max-parallelism=30 segment-costs=[222116656]
| 13:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales, RANDOM]
| HDFS partitions=1831/1831 files=1831 size=150.82MB
-| runtime filters: RF015[min_max] -> cs_item_sk, RF013[min_max] -> cs_promo_sk, RF009[min_max] -> cs_catalog_page_sk, RF014[bloom] -> cs_item_sk, RF012[bloom] -> cs_promo_sk, RF008[bloom] -> cs_catalog_page_sk
+| runtime filters: RF013[min_max] -> cs_item_sk, RF011[min_max] -> cs_promo_sk, RF009[min_max] -> cs_catalog_page_sk, RF014[bloom] -> cs_sold_date_sk, RF012[bloom] -> cs_item_sk, RF010[bloom] -> cs_promo_sk
| stored statistics:
| table: rows=4.32G size=280.96GB
| partitions: 1831/1831 rows=4.32G
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=21.52M
+| extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=31(filtered from 1831)
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=8 row-size=32B cardinality=431.79M(filtered from 4.32G) cost=1990691932
+| tuple-ids=8 row-size=32B cardinality=73.14M(filtered from 4.32G) cost=33703686
| in pipelines: 13(GETNEXT)
|
47:AGGREGATE [FINALIZE]
@@ -1034,13 +1038,13 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| in pipelines: 01(GETNEXT)
|
F02:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=61.72MB mem-reservation=2.00MB thread-reservation=1
-max-parallelism=390 segment-costs=[3880741788, 3667] cpu-comparison-result=287 [max(240 (self) vs 287 (sum children))]
+Per-Instance Resources: mem-estimate=58.62MB mem-reservation=2.00MB thread-reservation=1
+max-parallelism=180 segment-costs=[1764974287, 3667] cpu-comparison-result=287 [max(152 (self) vs 287 (sum children))]
12:AGGREGATE [STREAMING]
| output: sum(ss_ext_sales_price), sum(coalesce(sr_return_amt, CAST(0 AS DECIMAL(7,2)))), sum(ss_net_profit - coalesce(sr_net_loss, CAST(0 AS DECIMAL(7,2))))
| group by: s_store_id
| mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=6 row-size=76B cardinality=678 cost=369345185
+| tuple-ids=6 row-size=76B cardinality=678 cost=6485765
| in pipelines: 01(GETNEXT)
|
11:HASH JOIN [INNER JOIN, BROADCAST]
@@ -1048,16 +1052,16 @@ max-parallelism=390 segment-costs=[3880741788, 3667] cpu-comparison-result=287 [
| hash predicates: ss_store_sk = s_store_sk
| fk/pk conjuncts: ss_store_sk = s_store_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1N,4,5,2,3 row-size=117B cardinality=853.49M cost=373571195
+| tuple-ids=0,1N,2,4,5,3 row-size=117B cardinality=14.51M cost=6349072
| in pipelines: 01(GETNEXT), 04(OPEN)
|
|--F27:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.33MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | Per-Instance Resources: mem-estimate=23.33MB mem-reservation=23.25MB thread-reservation=1
| | max-parallelism=10 segment-costs=[3140]
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: s_store_sk
-| | runtime filters: RF000[bloom] <- s_store_sk, RF001[min_max] <- s_store_sk
+| | runtime filters: RF001[min_max] <- s_store_sk
| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=1350
| |
| 45:EXCHANGE [BROADCAST]
@@ -1080,65 +1084,27 @@ max-parallelism=390 segment-costs=[3880741788, 3667] cpu-comparison-result=287 [
|
10:HASH JOIN [INNER JOIN, BROADCAST]
| hash-table-id=01
-| hash predicates: ss_sold_date_sk = d_date_sk
-| fk/pk conjuncts: ss_sold_date_sk = d_date_sk
-| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1N,4,5,2 row-size=85B cardinality=853.49M cost=373571195
-| in pipelines: 01(GETNEXT), 03(OPEN)
-|
-|--F28:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
-| JOIN BUILD
-| | join-table-id=01 plan-id=02 cohort-id=01
-| | build expressions: d_date_sk
-| | runtime filters: RF002[bloom] <- d_date_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
-| |
-| 44:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=2 row-size=8B cardinality=7.30K cost=9700
-| | in pipelines: 03(GETNEXT)
-| |
-| F05:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
-| 03:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| HDFS partitions=1/1 files=1 size=2.17MB
-| predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| stored statistics:
-| table: rows=73.05K size=2.17MB
-| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=73.05K
-| parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
-| in pipelines: 03(GETNEXT)
-|
-09:HASH JOIN [INNER JOIN, BROADCAST]
-| hash-table-id=02
| hash predicates: ss_promo_sk = p_promo_sk
| fk/pk conjuncts: ss_promo_sk = p_promo_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1N,4,5 row-size=77B cardinality=853.49M cost=376155403
+| tuple-ids=0,1N,2,4,5 row-size=85B cardinality=14.51M cost=6392992
| in pipelines: 01(GETNEXT), 06(OPEN)
|
-|--F29:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=10
+|--F28:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=10
| | Per-Instance Resources: mem-estimate=24.30MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[4139]
| JOIN BUILD
-| | join-table-id=02 plan-id=03 cohort-id=01
+| | join-table-id=01 plan-id=02 cohort-id=01
| | build expressions: p_promo_sk
-| | runtime filters: RF004[bloom] <- p_promo_sk, RF005[min_max] <- p_promo_sk
+| | runtime filters: RF002[bloom] <- p_promo_sk, RF003[min_max] <- p_promo_sk
| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=1779
| |
-| 43:EXCHANGE [BROADCAST]
+| 44:EXCHANGE [BROADCAST]
| | mem-estimate=50.53KB mem-reservation=0B thread-reservation=0
| | tuple-ids=5 row-size=17B cardinality=1.78K cost=2360
| | in pipelines: 06(GETNEXT)
| |
-| F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| F05:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=16.08MB mem-reservation=32.00KB thread-reservation=1
| max-parallelism=1 segment-costs=[300]
| 06:SCAN HDFS [tpcds_partitioned_parquet_snap.promotion, RANDOM]
@@ -1154,29 +1120,29 @@ max-parallelism=390 segment-costs=[3880741788, 3667] cpu-comparison-result=287 [
| tuple-ids=5 row-size=17B cardinality=1.78K cost=180
| in pipelines: 06(GETNEXT)
|
-08:HASH JOIN [INNER JOIN, BROADCAST]
-| hash-table-id=03
+09:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=02
| hash predicates: ss_item_sk = i_item_sk
| fk/pk conjuncts: ss_item_sk = i_item_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
-| tuple-ids=0,1N,4 row-size=60B cardinality=863.56M(filtered from 863.56M) cost=377980972
+| tuple-ids=0,1N,2,4 row-size=68B cardinality=14.68M cost=40324146
| in pipelines: 01(GETNEXT), 05(OPEN)
|
-|--F30:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=10
+|--F29:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=10
| | Per-Instance Resources: mem-estimate=35.82MB mem-reservation=35.50MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[83840]
| JOIN BUILD
-| | join-table-id=03 plan-id=04 cohort-id=01
+| | join-table-id=02 plan-id=03 cohort-id=01
| | build expressions: i_item_sk
-| | runtime filters: RF006[bloom] <- i_item_sk, RF007[min_max] <- i_item_sk
+| | runtime filters: RF004[bloom] <- i_item_sk, RF005[min_max] <- i_item_sk
| | mem-estimate=34.50MB mem-reservation=34.50MB spill-buffer=128.00KB thread-reservation=0 cost=36000
| |
-| 42:EXCHANGE [BROADCAST]
+| 43:EXCHANGE [BROADCAST]
| | mem-estimate=329.25KB mem-reservation=0B thread-reservation=0
| | tuple-ids=4 row-size=8B cardinality=36.00K cost=47840
| | in pipelines: 05(GETNEXT)
| |
-| F03:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
+| F04:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=512.00KB thread-reservation=1
| max-parallelism=4 segment-costs=[53143]
| 05:SCAN HDFS [tpcds_partitioned_parquet_snap.item, RANDOM]
@@ -1192,17 +1158,55 @@ max-parallelism=390 segment-costs=[3880741788, 3667] cpu-comparison-result=287 [
| tuple-ids=4 row-size=8B cardinality=36.00K cost=51588
| in pipelines: 05(GETNEXT)
|
+08:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=03
+| hash predicates: ss_sold_date_sk = d_date_sk
+| fk/pk conjuncts: ss_sold_date_sk = d_date_sk
+| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,1N,2 row-size=60B cardinality=146.84M cost=64272307
+| in pipelines: 01(GETNEXT), 03(OPEN)
+|
+|--F30:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=10
+| | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[71]
+| JOIN BUILD
+| | join-table-id=03 plan-id=04 cohort-id=01
+| | build expressions: d_date_sk
+| | runtime filters: RF006[bloom] <- d_date_sk
+| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=31
+| |
+| 42:EXCHANGE [BROADCAST]
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=2 row-size=8B cardinality=31 cost=40
+| | in pipelines: 03(GETNEXT)
+| |
+| F03:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| max-parallelism=1 segment-costs=[12521]
+| 03:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| HDFS partitions=1/1 files=1 size=2.17MB
+| predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| stored statistics:
+| table: rows=73.05K size=2.17MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=73.05K
+| parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| tuple-ids=2 row-size=8B cardinality=31 cost=12520
+| in pipelines: 03(GETNEXT)
+|
07:HASH JOIN [LEFT OUTER JOIN, PARTITIONED]
| hash-table-id=04
| hash predicates: ss_item_sk = sr_item_sk, ss_ticket_number = sr_ticket_number
| fk/pk conjuncts: ss_item_sk = sr_item_sk, ss_ticket_number = sr_ticket_number
| mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,1N row-size=52B cardinality=863.56M(filtered from 8.64G) cost=1787059842
+| tuple-ids=0,1N row-size=52B cardinality=146.84M(filtered from 8.64G) cost=1603220974
| in pipelines: 01(GETNEXT), 02(OPEN)
|
|--F31:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=120
| | Per-Instance Resources: mem-estimate=354.94MB mem-reservation=34.00MB thread-reservation=1
-| | max-parallelism=390 segment-costs=[1039464723]
+| | max-parallelism=180 segment-costs=[1039464723]
| JOIN BUILD
| | join-table-id=04 plan-id=05 cohort-id=01
| | build expressions: sr_item_sk, sr_ticket_number
@@ -1219,7 +1223,7 @@ max-parallelism=390 segment-costs=[3880741788, 3667] cpu-comparison-result=287 [
| max-parallelism=190 segment-costs=[1806773027]
| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.store_returns, RANDOM]
| HDFS partitions=2004/2004 files=2004 size=33.63MB
-| runtime filters: RF007[min_max] -> tpcds_partitioned_parquet_snap.store_returns.sr_item_sk, RF006[bloom] -> tpcds_partitioned_parquet_snap.store_returns.sr_item_sk
+| runtime filters: RF005[min_max] -> tpcds_partitioned_parquet_snap.store_returns.sr_item_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.store_returns.sr_item_sk
| stored statistics:
| table: rows=863.99M size=48.14GB
| partitions: 2004/2004 rows=863.99M
@@ -1230,31 +1234,31 @@ max-parallelism=390 segment-costs=[3880741788, 3667] cpu-comparison-result=287 [
| in pipelines: 02(GETNEXT)
|
40:EXCHANGE [HASH(ss_item_sk,ss_ticket_number)]
-| mem-estimate=14.22MB mem-reservation=0B thread-reservation=0
-| tuple-ids=0 row-size=32B cardinality=863.56M(filtered from 8.64G) cost=223057996
+| mem-estimate=11.12MB mem-reservation=0B thread-reservation=0
+| tuple-ids=0 row-size=32B cardinality=146.84M(filtered from 8.64G) cost=37929031
| in pipelines: 01(GETNEXT)
|
-F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
-Per-Host Shared Resources: mem-estimate=4.00MB mem-reservation=4.00MB thread-reservation=0 runtime-filters-memory=4.00MB
+F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=32 (adjusted from 120)
+Per-Host Shared Resources: mem-estimate=3.00MB mem-reservation=3.00MB thread-reservation=0 runtime-filters-memory=3.00MB
Per-Instance Resources: mem-estimate=32.88MB mem-reservation=8.00MB thread-reservation=1
-max-parallelism=630 segment-costs=[6205817173]
+max-parallelism=32 segment-costs=[445926760]
01:SCAN HDFS [tpcds_partitioned_parquet_snap.store_sales, RANDOM]
HDFS partitions=1824/1824 files=1824 size=199.33MB
- runtime filters: RF007[min_max] -> ss_item_sk, RF005[min_max] -> ss_promo_sk, RF001[min_max] -> ss_store_sk, RF006[bloom] -> ss_item_sk, RF004[bloom] -> ss_promo_sk, RF000[bloom] -> ss_store_sk, RF002[bloom] -> ss_sold_date_sk
+ runtime filters: RF005[min_max] -> ss_item_sk, RF003[min_max] -> ss_promo_sk, RF001[min_max] -> ss_store_sk, RF006[bloom] -> ss_sold_date_sk, RF004[bloom] -> ss_item_sk, RF002[bloom] -> ss_promo_sk
stored statistics:
table: rows=8.64G size=389.90GB
partitions: 1824/1824 rows=8.64G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=390.22M
+ extrapolated-rows=disabled max-scan-range-rows=390.22M est-scan-range=32(filtered from 1824)
mem-estimate=16.00MB mem-reservation=8.00MB thread-reservation=0
- tuple-ids=0 row-size=32B cardinality=863.56M(filtered from 8.64G) cost=3981282136
+ tuple-ids=0 row-size=32B cardinality=146.84M(filtered from 8.64G) cost=67664334
in pipelines: 01(GETNEXT)
---- PARALLELPLANS
-Max Per-Host Resource Reservation: Memory=1.78GB Threads=169
-Per-Host Resource Estimates: Memory=12.56GB
+Max Per-Host Resource Reservation: Memory=1.34GB Threads=126
+Per-Host Resource Estimates: Memory=10.89GB
F26:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=4.07MB mem-reservation=4.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[539] cpu-comparison-result=774 [max(1 (self) vs 774 (sum children))]
+| max-parallelism=1 segment-costs=[539] cpu-comparison-result=671 [max(1 (self) vs 671 (sum children))]
PLAN-ROOT SINK
| output exprs: CASE valid_tid(26,27,28) WHEN 26 THEN channel WHEN 27 THEN channel WHEN 28 THEN NULL END, CASE valid_tid(26,27,28) WHEN 26 THEN id WHEN 27 THEN NULL WHEN 28 THEN NULL END, aggif(valid_tid(26,27,28) IN (26, 27, 28), CASE valid_tid(26,27,28) WHEN 26 THEN sum(sales) WHEN 27 THEN sum(sales) WHEN 28 THEN sum(sales) END), aggif(valid_tid(26,27,28) IN (26, 27, 28), CASE valid_tid(26,27,28) WHEN 26 THEN sum(`returns`) WHEN 27 THEN sum(`returns`) WHEN 28 THEN sum(`returns`) END), [...]
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=500
@@ -1268,7 +1272,7 @@ PLAN-ROOT SINK
|
F25:PLAN FRAGMENT [HASH(CASE valid_tid(26,27,28) WHEN 26 THEN murmur_hash(channel) WHEN 27 THEN murmur_hash(channel) WHEN 28 THEN murmur_hash(NULL) END,CASE valid_tid(26,27,28) WHEN 26 THEN murmur_hash(id) WHEN 27 THEN murmur_hash(NULL) WHEN 28 THEN murmur_hash(NULL) END)] hosts=10 instances=10 (adjusted from 120)
Per-Instance Resources: mem-estimate=57.48MB mem-reservation=7.75MB thread-reservation=1
-max-parallelism=10 segment-costs=[265114, 221854, 313895, 220] cpu-comparison-result=774 [max(10 (self) vs 774 (sum children))]
+max-parallelism=10 segment-costs=[265114, 221854, 313895, 220] cpu-comparison-result=671 [max(10 (self) vs 671 (sum children))]
39:TOP-N [LIMIT=100]
| order by: CASE valid_tid(26,27,28) WHEN 26 THEN channel WHEN 27 THEN channel WHEN 28 THEN NULL END ASC, CASE valid_tid(26,27,28) WHEN 26 THEN id WHEN 27 THEN NULL WHEN 28 THEN NULL END ASC
| mem-estimate=7.03KB mem-reservation=0B thread-reservation=0
@@ -1302,8 +1306,8 @@ max-parallelism=10 segment-costs=[265114, 221854, 313895, 220] cpu-comparison-re
| in pipelines: 47(GETNEXT), 55(GETNEXT), 63(GETNEXT)
|
F24:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=58.54MB mem-reservation=7.94MB thread-reservation=1
-max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-comparison-result=774 [max(120 (self) vs 774 (sum children))]
+Per-Instance Resources: mem-estimate=56.20MB mem-reservation=7.94MB thread-reservation=1
+max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-comparison-result=671 [max(120 (self) vs 671 (sum children))]
37:AGGREGATE [STREAMING]
| Class 0
| output: sum(sales), sum(returns), sum(profit)
@@ -1331,80 +1335,43 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | in pipelines: 63(GETNEXT), 25(OPEN)
| |
| 62:EXCHANGE [HASH(web_site_id)]
-| | mem-estimate=258.06KB mem-reservation=0B thread-reservation=0
+| | mem-estimate=129.15KB mem-reservation=0B thread-reservation=0
| | tuple-ids=22 row-size=76B cardinality=33 cost=15
| | in pipelines: 25(GETNEXT)
| |
-| F18:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=100 (adjusted from 120)
-| Per-Instance Resources: mem-estimate=61.72MB mem-reservation=2.00MB thread-reservation=1
-| max-parallelism=100 segment-costs=[966479381, 178] cpu-comparison-result=220 [max(220 (self) vs 197 (sum children))]
+| F18:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=50 (adjusted from 120)
+| Per-Instance Resources: mem-estimate=58.20MB mem-reservation=2.00MB thread-reservation=1
+| max-parallelism=50 segment-costs=[441159182, 178] cpu-comparison-result=147 [max(70 (self) vs 147 (sum children))]
| 36:AGGREGATE [STREAMING]
| | output: sum(ws_ext_sales_price), sum(coalesce(wr_return_amt, CAST(0 AS DECIMAL(7,2)))), sum(ws_net_profit - coalesce(wr_net_loss, CAST(0 AS DECIMAL(7,2))))
| | group by: web_site_id
| | mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=22 row-size=76B cardinality=33 cost=90915562
+| | tuple-ids=22 row-size=76B cardinality=33 cost=1555315
| | in pipelines: 25(GETNEXT)
| |
| 35:HASH JOIN [INNER JOIN, BROADCAST]
| | hash-table-id=10
-| | hash predicates: ws_sold_date_sk = d_date_sk
-| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=16,17N,20,19,21,18 row-size=117B cardinality=210.19M cost=91998202
-| | in pipelines: 25(GETNEXT), 27(OPEN)
-| |
-| |--F37:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=19.44MB mem-reservation=19.38MB thread-reservation=1
-| | | max-parallelism=10 segment-costs=[17005]
-| | JOIN BUILD
-| | | join-table-id=10 plan-id=11 cohort-id=03
-| | | build expressions: d_date_sk
-| | | mem-estimate=19.38MB mem-reservation=19.38MB spill-buffer=64.00KB thread-reservation=0 cost=7305
-| | |
-| | 61:EXCHANGE [BROADCAST]
-| | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=18 row-size=8B cardinality=7.30K cost=9700
-| | | in pipelines: 27(GETNEXT)
-| | |
-| | F22:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[12835]
-| | 27:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| | HDFS partitions=1/1 files=1 size=2.17MB
-| | predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | stored statistics:
-| | table: rows=73.05K size=2.17MB
-| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=73.05K
-| | parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=18 row-size=8B cardinality=7.30K cost=12520
-| | in pipelines: 27(GETNEXT)
-| |
-| 34:HASH JOIN [INNER JOIN, BROADCAST]
-| | hash-table-id=11
| | hash predicates: ws_promo_sk = p_promo_sk
| | fk/pk conjuncts: ws_promo_sk = p_promo_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=16,17N,20,19,21 row-size=109B cardinality=210.19M cost=92634607
+| | tuple-ids=16,17N,18,20,19,21 row-size=117B cardinality=3.57M cost=1574382
| | in pipelines: 25(GETNEXT), 30(OPEN)
| |
-| |--F38:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=20.42MB mem-reservation=20.38MB thread-reservation=1 runtime-filters-memory=1.00MB
+| |--F37:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=10
+| | | Per-Instance Resources: mem-estimate=10.74MB mem-reservation=10.69MB thread-reservation=1 runtime-filters-memory=1.00MB
| | | max-parallelism=10 segment-costs=[4139]
| | JOIN BUILD
-| | | join-table-id=11 plan-id=12 cohort-id=03
+| | | join-table-id=10 plan-id=11 cohort-id=03
| | | build expressions: p_promo_sk
-| | | runtime filters: RF018[bloom] <- p_promo_sk, RF019[min_max] <- p_promo_sk
-| | | mem-estimate=19.38MB mem-reservation=19.38MB spill-buffer=64.00KB thread-reservation=0 cost=1779
+| | | runtime filters: RF016[bloom] <- p_promo_sk, RF017[min_max] <- p_promo_sk
+| | | mem-estimate=9.69MB mem-reservation=9.69MB spill-buffer=64.00KB thread-reservation=0 cost=1779
| | |
-| | 60:EXCHANGE [BROADCAST]
+| | 61:EXCHANGE [BROADCAST]
| | | mem-estimate=50.53KB mem-reservation=0B thread-reservation=0
| | | tuple-ids=21 row-size=17B cardinality=1.78K cost=2360
| | | in pipelines: 30(GETNEXT)
| | |
-| | F21:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | F22:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| | Per-Instance Resources: mem-estimate=16.08MB mem-reservation=32.00KB thread-reservation=1
| | max-parallelism=1 segment-costs=[300]
| | 30:SCAN HDFS [tpcds_partitioned_parquet_snap.promotion, RANDOM]
@@ -1420,29 +1387,29 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | tuple-ids=21 row-size=17B cardinality=1.78K cost=180
| | in pipelines: 30(GETNEXT)
| |
-| 33:HASH JOIN [INNER JOIN, BROADCAST]
-| | hash-table-id=12
+| 34:HASH JOIN [INNER JOIN, BROADCAST]
+| | hash-table-id=11
| | hash predicates: ws_web_site_sk = web_site_sk
| | fk/pk conjuncts: ws_web_site_sk = web_site_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=16,17N,20,19 row-size=92B cardinality=212.67M cost=93910684
+| | tuple-ids=16,17N,18,20,19 row-size=100B cardinality=3.61M cost=1596070
| | in pipelines: 25(GETNEXT), 28(OPEN)
| |
-| |--F39:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=20.39MB mem-reservation=20.38MB thread-reservation=1 runtime-filters-memory=1.00MB
+| |--F38:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=10
+| | | Per-Instance Resources: mem-estimate=10.70MB mem-reservation=10.69MB thread-reservation=1 runtime-filters-memory=1.00MB
| | | max-parallelism=10 segment-costs=[146]
| | JOIN BUILD
-| | | join-table-id=12 plan-id=13 cohort-id=03
+| | | join-table-id=11 plan-id=12 cohort-id=03
| | | build expressions: web_site_sk
-| | | runtime filters: RF020[bloom] <- web_site_sk, RF021[min_max] <- web_site_sk
-| | | mem-estimate=19.38MB mem-reservation=19.38MB spill-buffer=64.00KB thread-reservation=0 cost=66
+| | | runtime filters: RF018[bloom] <- web_site_sk, RF019[min_max] <- web_site_sk
+| | | mem-estimate=9.69MB mem-reservation=9.69MB spill-buffer=64.00KB thread-reservation=0 cost=66
| | |
-| | 59:EXCHANGE [BROADCAST]
+| | 60:EXCHANGE [BROADCAST]
| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
| | | tuple-ids=19 row-size=32B cardinality=66 cost=80
| | | in pipelines: 28(GETNEXT)
| | |
-| | F20:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | F21:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| | Per-Instance Resources: mem-estimate=16.14MB mem-reservation=16.00KB thread-reservation=1
| | max-parallelism=1 segment-costs=[26]
| | 28:SCAN HDFS [tpcds_partitioned_parquet_snap.web_site, RANDOM]
@@ -1455,29 +1422,29 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | tuple-ids=19 row-size=32B cardinality=66 cost=19
| | in pipelines: 28(GETNEXT)
| |
-| 32:HASH JOIN [INNER JOIN, BROADCAST]
-| | hash-table-id=13
+| 33:HASH JOIN [INNER JOIN, BROADCAST]
+| | hash-table-id=12
| | hash predicates: ws_item_sk = i_item_sk
| | fk/pk conjuncts: ws_item_sk = i_item_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
-| | tuple-ids=16,17N,20 row-size=60B cardinality=215.89M(filtered from 215.89M) cost=94494550
+| | tuple-ids=16,17N,18,20 row-size=68B cardinality=3.67M cost=10080962
| | in pipelines: 25(GETNEXT), 29(OPEN)
| |
-| |--F40:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=30.07MB mem-reservation=29.75MB thread-reservation=1 runtime-filters-memory=1.00MB
+| |--F39:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=10
+| | | Per-Instance Resources: mem-estimate=15.70MB mem-reservation=15.38MB thread-reservation=1 runtime-filters-memory=1.00MB
| | | max-parallelism=10 segment-costs=[83840]
| | JOIN BUILD
-| | | join-table-id=13 plan-id=14 cohort-id=03
+| | | join-table-id=12 plan-id=13 cohort-id=03
| | | build expressions: i_item_sk
-| | | runtime filters: RF022[bloom] <- i_item_sk, RF023[min_max] <- i_item_sk
-| | | mem-estimate=28.75MB mem-reservation=28.75MB spill-buffer=128.00KB thread-reservation=0 cost=36000
+| | | runtime filters: RF020[bloom] <- i_item_sk, RF021[min_max] <- i_item_sk
+| | | mem-estimate=14.38MB mem-reservation=14.38MB spill-buffer=128.00KB thread-reservation=0 cost=36000
| | |
-| | 58:EXCHANGE [BROADCAST]
+| | 59:EXCHANGE [BROADCAST]
| | | mem-estimate=329.25KB mem-reservation=0B thread-reservation=0
| | | tuple-ids=20 row-size=8B cardinality=36.00K cost=47840
| | | in pipelines: 29(GETNEXT)
| | |
-| | F19:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
+| | F20:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=512.00KB thread-reservation=1
| | max-parallelism=4 segment-costs=[53143]
| | 29:SCAN HDFS [tpcds_partitioned_parquet_snap.item, RANDOM]
@@ -1493,21 +1460,59 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | tuple-ids=20 row-size=8B cardinality=36.00K cost=51588
| | in pipelines: 29(GETNEXT)
| |
+| 32:HASH JOIN [INNER JOIN, BROADCAST]
+| | hash-table-id=13
+| | hash predicates: ws_sold_date_sk = d_date_sk
+| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
+| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=16,17N,18 row-size=60B cardinality=36.71M cost=16067959
+| | in pipelines: 25(GETNEXT), 27(OPEN)
+| |
+| |--F40:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=10
+| | | Per-Instance Resources: mem-estimate=10.70MB mem-reservation=10.69MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[71]
+| | JOIN BUILD
+| | | join-table-id=13 plan-id=14 cohort-id=03
+| | | build expressions: d_date_sk
+| | | runtime filters: RF022[bloom] <- d_date_sk
+| | | mem-estimate=9.69MB mem-reservation=9.69MB spill-buffer=64.00KB thread-reservation=0 cost=31
+| | |
+| | 58:EXCHANGE [BROADCAST]
+| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=18 row-size=8B cardinality=31 cost=40
+| | | in pipelines: 27(GETNEXT)
+| | |
+| | F19:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| | max-parallelism=1 segment-costs=[12521]
+| | 27:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| | HDFS partitions=1/1 files=1 size=2.17MB
+| | predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | stored statistics:
+| | table: rows=73.05K size=2.17MB
+| | columns: all
+| | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | tuple-ids=18 row-size=8B cardinality=31 cost=12520
+| | in pipelines: 27(GETNEXT)
+| |
| 31:HASH JOIN [LEFT OUTER JOIN, PARTITIONED]
| | hash-table-id=14
| | hash predicates: ws_item_sk = wr_item_sk, ws_order_number = wr_order_number
| | fk/pk conjuncts: ws_item_sk = wr_item_sk, ws_order_number = wr_order_number
| | mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| | tuple-ids=16,17N row-size=52B cardinality=215.89M(filtered from 2.16G) cost=446761686
+| | tuple-ids=16,17N row-size=52B cardinality=36.71M(filtered from 2.16G) cost=400802306
| | in pipelines: 25(GETNEXT), 26(OPEN)
| |
-| |--F41:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=100 (adjusted from 120)
-| | | Per-Instance Resources: mem-estimate=113.81MB mem-reservation=34.00MB thread-reservation=1
-| | | max-parallelism=100 segment-costs=[259872831]
+| |--F41:PLAN FRAGMENT [HASH(ws_item_sk,ws_order_number)] hosts=10 instances=50 (adjusted from 120)
+| | | Per-Instance Resources: mem-estimate=216.45MB mem-reservation=34.00MB thread-reservation=1
+| | | max-parallelism=50 segment-costs=[259872831]
| | JOIN BUILD
| | | join-table-id=14 plan-id=15 cohort-id=03
| | | build expressions: wr_item_sk, wr_order_number
-| | | mem-estimate=102.64MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 cost=216002686
+| | | mem-estimate=205.28MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 cost=216002686
| | |
| | 57:EXCHANGE [HASH(wr_item_sk,wr_order_number)]
| | | mem-estimate=11.17MB mem-reservation=0B thread-reservation=0
@@ -1516,11 +1521,11 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | |
| | F17:PLAN FRAGMENT [RANDOM] hosts=10 instances=50 (adjusted from 120)
| | Per-Host Shared Resources: mem-estimate=1.00MB mem-reservation=1.00MB thread-reservation=0 runtime-filters-memory=1.00MB
-| | Per-Instance Resources: mem-estimate=25.38MB mem-reservation=256.00KB thread-reservation=1
+| | Per-Instance Resources: mem-estimate=20.69MB mem-reservation=256.00KB thread-reservation=1
| | max-parallelism=50 segment-costs=[451704816]
| | 26:SCAN HDFS [tpcds_partitioned_parquet_snap.web_returns, RANDOM]
| | HDFS partitions=2114/2114 files=2114 size=20.70MB
-| | runtime filters: RF023[min_max] -> tpcds_partitioned_parquet_snap.web_returns.wr_item_sk, RF022[bloom] -> tpcds_partitioned_parquet_snap.web_returns.wr_item_sk
+| | runtime filters: RF021[min_max] -> tpcds_partitioned_parquet_snap.web_returns.wr_item_sk, RF020[bloom] -> tpcds_partitioned_parquet_snap.web_returns.wr_item_sk
| | stored statistics:
| | table: rows=216.00M size=16.74GB
| | partitions: 2114/2114 rows=216.00M
@@ -1531,24 +1536,24 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | in pipelines: 26(GETNEXT)
| |
| 56:EXCHANGE [HASH(ws_item_sk,ws_order_number)]
-| | mem-estimate=14.22MB mem-reservation=0B thread-reservation=0
-| | tuple-ids=16 row-size=32B cardinality=215.89M(filtered from 2.16G) cost=55764090
+| | mem-estimate=10.70MB mem-reservation=0B thread-reservation=0
+| | tuple-ids=16 row-size=32B cardinality=36.71M(filtered from 2.16G) cost=9482188
| | in pipelines: 25(GETNEXT)
| |
-| F16:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
-| Per-Host Shared Resources: mem-estimate=3.00MB mem-reservation=3.00MB thread-reservation=0 runtime-filters-memory=3.00MB
-| Per-Instance Resources: mem-estimate=30.06MB mem-reservation=128.00KB thread-reservation=1
-| max-parallelism=160 segment-costs=[1551442921]
+| F16:PLAN FRAGMENT [RANDOM] hosts=10 instances=20 (adjusted from 120)
+| Per-Host Shared Resources: mem-estimate=4.00MB mem-reservation=4.00MB thread-reservation=0 runtime-filters-memory=4.00MB
+| Per-Instance Resources: mem-estimate=23.03MB mem-reservation=128.00KB thread-reservation=1
+| max-parallelism=20 segment-costs=[111480872]
| 25:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales, RANDOM]
| HDFS partitions=1824/1824 files=1824 size=84.18MB
-| runtime filters: RF023[min_max] -> ws_item_sk, RF021[min_max] -> ws_web_site_sk, RF019[min_max] -> ws_promo_sk, RF022[bloom] -> ws_item_sk, RF020[bloom] -> ws_web_site_sk, RF018[bloom] -> ws_promo_sk
+| runtime filters: RF021[min_max] -> ws_item_sk, RF019[min_max] -> ws_web_site_sk, RF017[min_max] -> ws_promo_sk, RF022[bloom] -> ws_sold_date_sk, RF020[bloom] -> ws_item_sk, RF018[bloom] -> ws_web_site_sk, RF016[bloom] -> ws_promo_sk
| stored statistics:
| table: rows=2.16G size=145.75GB
| partitions: 1824/1824 rows=2.16G
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=3.37M
+| extrapolated-rows=disabled max-scan-range-rows=3.37M est-scan-range=32(filtered from 1824)
| mem-estimate=16.00MB mem-reservation=128.00KB thread-reservation=0
-| tuple-ids=16 row-size=32B cardinality=215.89M(filtered from 2.16G) cost=995313239
+| tuple-ids=16 row-size=32B cardinality=36.71M(filtered from 2.16G) cost=16915959
| in pipelines: 25(GETNEXT)
|
|--55:AGGREGATE [FINALIZE]
@@ -1559,18 +1564,18 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | in pipelines: 55(GETNEXT), 13(OPEN)
| |
| 54:EXCHANGE [HASH(cp_catalog_page_id)]
-| | mem-estimate=9.64MB mem-reservation=0B thread-reservation=0
+| | mem-estimate=7.29MB mem-reservation=0B thread-reservation=0
| | tuple-ids=14 row-size=76B cardinality=36.00K cost=16585
| | in pipelines: 13(GETNEXT)
| |
-| F10:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=120
-| Per-Instance Resources: mem-estimate=61.72MB mem-reservation=5.00MB thread-reservation=1
-| max-parallelism=200 segment-costs=[1951577513, 194745] cpu-comparison-result=267 [max(240 (self) vs 267 (sum children))]
+| F10:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=90 (adjusted from 120)
+| Per-Instance Resources: mem-estimate=58.55MB mem-reservation=5.00MB thread-reservation=1
+| max-parallelism=90 segment-costs=[891170974, 194745] cpu-comparison-result=237 [max(120 (self) vs 237 (sum children))]
| 24:AGGREGATE [STREAMING]
| | output: sum(cs_ext_sales_price), sum(coalesce(cr_return_amount, CAST(0 AS DECIMAL(7,2)))), sum(cs_net_profit - coalesce(cr_net_loss, CAST(0 AS DECIMAL(7,2))))
| | group by: cp_catalog_page_id
| | mem-estimate=10.00MB mem-reservation=5.00MB spill-buffer=256.00KB thread-reservation=0
-| | tuple-ids=14 row-size=76B cardinality=36.00K cost=195834351
+| | tuple-ids=14 row-size=76B cardinality=36.00K cost=12273052
| | in pipelines: 13(GETNEXT)
| |
| 23:HASH JOIN [INNER JOIN, BROADCAST]
@@ -1578,17 +1583,17 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | hash predicates: cs_catalog_page_sk = cp_catalog_page_sk
| | fk/pk conjuncts: cs_catalog_page_sk = cp_catalog_page_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
-| | tuple-ids=8,9N,12,13,10,11 row-size=117B cardinality=426.75M cost=186790370
+| | tuple-ids=8,9N,10,12,13,11 row-size=117B cardinality=7.23M cost=3162480
| | in pipelines: 13(GETNEXT), 16(OPEN)
| |
| |--F32:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=36.63MB mem-reservation=35.50MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | Per-Instance Resources: mem-estimate=27.01MB mem-reservation=25.88MB thread-reservation=1
| | | max-parallelism=10 segment-costs=[83840]
| | JOIN BUILD
| | | join-table-id=05 plan-id=06 cohort-id=02
| | | build expressions: cp_catalog_page_sk
-| | | runtime filters: RF008[bloom] <- cp_catalog_page_sk, RF009[min_max] <- cp_catalog_page_sk
-| | | mem-estimate=34.50MB mem-reservation=34.50MB spill-buffer=128.00KB thread-reservation=0 cost=36000
+| | | runtime filters: RF009[min_max] <- cp_catalog_page_sk
+| | | mem-estimate=25.88MB mem-reservation=25.88MB spill-buffer=128.00KB thread-reservation=0 cost=36000
| | |
| | 53:EXCHANGE [BROADCAST]
| | | mem-estimate=1.13MB mem-reservation=0B thread-reservation=0
@@ -1610,64 +1615,27 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| |
| 22:HASH JOIN [INNER JOIN, BROADCAST]
| | hash-table-id=06
-| | hash predicates: cs_sold_date_sk = d_date_sk
-| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
-| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=8,9N,12,13,10 row-size=85B cardinality=426.75M cost=186790370
-| | in pipelines: 13(GETNEXT), 15(OPEN)
-| |
-| |--F33:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=23.32MB mem-reservation=23.25MB thread-reservation=1
-| | | max-parallelism=10 segment-costs=[17005]
-| | JOIN BUILD
-| | | join-table-id=06 plan-id=07 cohort-id=02
-| | | build expressions: d_date_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
-| | |
-| | 52:EXCHANGE [BROADCAST]
-| | | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | | tuple-ids=10 row-size=8B cardinality=7.30K cost=9700
-| | | in pipelines: 15(GETNEXT)
-| | |
-| | F13:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| | max-parallelism=1 segment-costs=[12835]
-| | 15:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| | HDFS partitions=1/1 files=1 size=2.17MB
-| | predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | stored statistics:
-| | table: rows=73.05K size=2.17MB
-| | columns: all
-| | extrapolated-rows=disabled max-scan-range-rows=73.05K
-| | parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=10 row-size=8B cardinality=7.30K cost=12520
-| | in pipelines: 15(GETNEXT)
-| |
-| 21:HASH JOIN [INNER JOIN, BROADCAST]
-| | hash-table-id=07
| | hash predicates: cs_promo_sk = p_promo_sk
| | fk/pk conjuncts: cs_promo_sk = p_promo_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=8,9N,12,13 row-size=77B cardinality=426.75M cost=188082507
+| | tuple-ids=8,9N,10,12,13 row-size=85B cardinality=7.23M cost=3184357
| | in pipelines: 13(GETNEXT), 18(OPEN)
| |
-| |--F34:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=24.30MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| |--F33:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
+| | | Per-Instance Resources: mem-estimate=18.49MB mem-reservation=18.44MB thread-reservation=1 runtime-filters-memory=1.00MB
| | | max-parallelism=10 segment-costs=[4139]
| | JOIN BUILD
-| | | join-table-id=07 plan-id=08 cohort-id=02
+| | | join-table-id=06 plan-id=07 cohort-id=02
| | | build expressions: p_promo_sk
-| | | runtime filters: RF012[bloom] <- p_promo_sk, RF013[min_max] <- p_promo_sk
-| | | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=1779
+| | | runtime filters: RF010[bloom] <- p_promo_sk, RF011[min_max] <- p_promo_sk
+| | | mem-estimate=17.44MB mem-reservation=17.44MB spill-buffer=64.00KB thread-reservation=0 cost=1779
| | |
-| | 51:EXCHANGE [BROADCAST]
+| | 52:EXCHANGE [BROADCAST]
| | | mem-estimate=50.53KB mem-reservation=0B thread-reservation=0
| | | tuple-ids=13 row-size=17B cardinality=1.78K cost=2360
| | | in pipelines: 18(GETNEXT)
| | |
-| | F12:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | F13:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| | Per-Instance Resources: mem-estimate=16.08MB mem-reservation=32.00KB thread-reservation=1
| | max-parallelism=1 segment-costs=[300]
| | 18:SCAN HDFS [tpcds_partitioned_parquet_snap.promotion, RANDOM]
@@ -1683,29 +1651,29 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | tuple-ids=13 row-size=17B cardinality=1.78K cost=180
| | in pipelines: 18(GETNEXT)
| |
-| 20:HASH JOIN [INNER JOIN, BROADCAST]
-| | hash-table-id=08
+| 21:HASH JOIN [INNER JOIN, BROADCAST]
+| | hash-table-id=07
| | hash predicates: cs_item_sk = i_item_sk
| | fk/pk conjuncts: cs_item_sk = i_item_sk
| | mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
-| | tuple-ids=8,9N,12 row-size=60B cardinality=431.79M cost=188995315
+| | tuple-ids=8,9N,10,12 row-size=68B cardinality=7.31M cost=20085506
| | in pipelines: 13(GETNEXT), 17(OPEN)
| |
-| |--F35:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
-| | | Per-Instance Resources: mem-estimate=35.82MB mem-reservation=35.50MB thread-reservation=1 runtime-filters-memory=1.00MB
+| |--F34:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
+| | | Per-Instance Resources: mem-estimate=27.20MB mem-reservation=26.88MB thread-reservation=1 runtime-filters-memory=1.00MB
| | | max-parallelism=10 segment-costs=[83840]
| | JOIN BUILD
-| | | join-table-id=08 plan-id=09 cohort-id=02
+| | | join-table-id=07 plan-id=08 cohort-id=02
| | | build expressions: i_item_sk
-| | | runtime filters: RF014[bloom] <- i_item_sk, RF015[min_max] <- i_item_sk
-| | | mem-estimate=34.50MB mem-reservation=34.50MB spill-buffer=128.00KB thread-reservation=0 cost=36000
+| | | runtime filters: RF012[bloom] <- i_item_sk, RF013[min_max] <- i_item_sk
+| | | mem-estimate=25.88MB mem-reservation=25.88MB spill-buffer=128.00KB thread-reservation=0 cost=36000
| | |
-| | 50:EXCHANGE [BROADCAST]
+| | 51:EXCHANGE [BROADCAST]
| | | mem-estimate=329.25KB mem-reservation=0B thread-reservation=0
| | | tuple-ids=12 row-size=8B cardinality=36.00K cost=47840
| | | in pipelines: 17(GETNEXT)
| | |
-| | F11:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
+| | F12:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=512.00KB thread-reservation=1
| | max-parallelism=4 segment-costs=[53143]
| | 17:SCAN HDFS [tpcds_partitioned_parquet_snap.item, RANDOM]
@@ -1721,21 +1689,59 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | tuple-ids=12 row-size=8B cardinality=36.00K cost=51588
| | in pipelines: 17(GETNEXT)
| |
+| 20:HASH JOIN [INNER JOIN, BROADCAST]
+| | hash-table-id=08
+| | hash predicates: cs_sold_date_sk = d_date_sk
+| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
+| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=8,9N,10 row-size=60B cardinality=73.14M cost=32014113
+| | in pipelines: 13(GETNEXT), 15(OPEN)
+| |
+| |--F35:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=10
+| | | Per-Instance Resources: mem-estimate=18.45MB mem-reservation=18.44MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | | max-parallelism=10 segment-costs=[71]
+| | JOIN BUILD
+| | | join-table-id=08 plan-id=09 cohort-id=02
+| | | build expressions: d_date_sk
+| | | runtime filters: RF014[bloom] <- d_date_sk
+| | | mem-estimate=17.44MB mem-reservation=17.44MB spill-buffer=64.00KB thread-reservation=0 cost=31
+| | |
+| | 50:EXCHANGE [BROADCAST]
+| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=10 row-size=8B cardinality=31 cost=40
+| | | in pipelines: 15(GETNEXT)
+| | |
+| | F11:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| | max-parallelism=1 segment-costs=[12521]
+| | 15:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| | HDFS partitions=1/1 files=1 size=2.17MB
+| | predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | stored statistics:
+| | table: rows=73.05K size=2.17MB
+| | columns: all
+| | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | tuple-ids=10 row-size=8B cardinality=31 cost=12520
+| | in pipelines: 15(GETNEXT)
+| |
| 19:HASH JOIN [LEFT OUTER JOIN, PARTITIONED]
| | hash-table-id=09
| | hash predicates: cs_item_sk = cr_item_sk, cs_order_number = cr_order_number
| | fk/pk conjuncts: cs_item_sk = cr_item_sk, cs_order_number = cr_order_number
| | mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| | tuple-ids=8,9N row-size=52B cardinality=431.79M(filtered from 4.32G) cost=893552752
+| | tuple-ids=8,9N row-size=52B cardinality=73.14M(filtered from 4.32G) cost=801558970
| | in pipelines: 13(GETNEXT), 14(OPEN)
| |
-| |--F36:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=120
-| | | Per-Instance Resources: mem-estimate=183.41MB mem-reservation=34.00MB thread-reservation=1
-| | | max-parallelism=200 segment-costs=[519759641]
+| |--F36:PLAN FRAGMENT [HASH(cs_item_sk,cs_order_number)] hosts=10 instances=90 (adjusted from 120)
+| | | Per-Instance Resources: mem-estimate=240.43MB mem-reservation=34.00MB thread-reservation=1
+| | | max-parallelism=90 segment-costs=[519759641]
| | JOIN BUILD
| | | join-table-id=09 plan-id=10 cohort-id=02
| | | build expressions: cr_item_sk, cr_order_number
-| | | mem-estimate=171.07MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 cost=432016991
+| | | mem-estimate=228.09MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 cost=432016991
| | |
| | 49:EXCHANGE [HASH(cr_item_sk,cr_order_number)]
| | | mem-estimate=12.34MB mem-reservation=0B thread-reservation=0
@@ -1744,11 +1750,11 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | |
| | F09:PLAN FRAGMENT [RANDOM] hosts=10 instances=100 (adjusted from 120)
| | Per-Host Shared Resources: mem-estimate=1.00MB mem-reservation=1.00MB thread-reservation=0 runtime-filters-memory=1.00MB
-| | Per-Instance Resources: mem-estimate=27.25MB mem-reservation=32.00KB thread-reservation=1
+| | Per-Instance Resources: mem-estimate=24.44MB mem-reservation=32.00KB thread-reservation=1
| | max-parallelism=100 segment-costs=[903433931]
| | 14:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_returns, RANDOM]
| | HDFS partitions=2060/2060 files=2060 size=30.47MB
-| | runtime filters: RF015[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk, RF014[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk
+| | runtime filters: RF013[min_max] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk, RF012[bloom] -> tpcds_partitioned_parquet_snap.catalog_returns.cr_item_sk
| | stored statistics:
| | table: rows=432.02M size=32.77GB
| | partitions: 2060/2060 rows=432.02M
@@ -1759,24 +1765,24 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| | in pipelines: 14(GETNEXT)
| |
| 48:EXCHANGE [HASH(cs_item_sk,cs_order_number)]
-| | mem-estimate=14.22MB mem-reservation=0B thread-reservation=0
-| | tuple-ids=8 row-size=32B cardinality=431.79M(filtered from 4.32G) cost=111531848
+| | mem-estimate=11.05MB mem-reservation=0B thread-reservation=0
+| | tuple-ids=8 row-size=32B cardinality=73.14M(filtered from 4.32G) cost=18892496
| | in pipelines: 13(GETNEXT)
| |
-| F08:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
+| F08:PLAN FRAGMENT [RANDOM] hosts=10 instances=30 (adjusted from 120)
| Per-Host Shared Resources: mem-estimate=3.00MB mem-reservation=3.00MB thread-reservation=0 runtime-filters-memory=3.00MB
-| Per-Instance Resources: mem-estimate=32.88MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=320 segment-costs=[3102987871]
+| Per-Instance Resources: mem-estimate=28.66MB mem-reservation=1.00MB thread-reservation=1
+| max-parallelism=30 segment-costs=[222116656]
| 13:SCAN HDFS [tpcds_partitioned_parquet_snap.catalog_sales, RANDOM]
| HDFS partitions=1831/1831 files=1831 size=150.82MB
-| runtime filters: RF015[min_max] -> cs_item_sk, RF013[min_max] -> cs_promo_sk, RF009[min_max] -> cs_catalog_page_sk, RF014[bloom] -> cs_item_sk, RF012[bloom] -> cs_promo_sk, RF008[bloom] -> cs_catalog_page_sk
+| runtime filters: RF013[min_max] -> cs_item_sk, RF011[min_max] -> cs_promo_sk, RF009[min_max] -> cs_catalog_page_sk, RF014[bloom] -> cs_sold_date_sk, RF012[bloom] -> cs_item_sk, RF010[bloom] -> cs_promo_sk
| stored statistics:
| table: rows=4.32G size=280.96GB
| partitions: 1831/1831 rows=4.32G
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=21.52M
+| extrapolated-rows=disabled max-scan-range-rows=21.52M est-scan-range=31(filtered from 1831)
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=8 row-size=32B cardinality=431.79M(filtered from 4.32G) cost=1990691932
+| tuple-ids=8 row-size=32B cardinality=73.14M(filtered from 4.32G) cost=33703686
| in pipelines: 13(GETNEXT)
|
47:AGGREGATE [FINALIZE]
@@ -1792,13 +1798,13 @@ max-parallelism=120 segment-costs=[2169, 125388, 29, 344492, 567467] cpu-compari
| in pipelines: 01(GETNEXT)
|
F02:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=61.72MB mem-reservation=2.00MB thread-reservation=1
-max-parallelism=390 segment-costs=[3880741788, 3667] cpu-comparison-result=287 [max(240 (self) vs 287 (sum children))]
+Per-Instance Resources: mem-estimate=58.62MB mem-reservation=2.00MB thread-reservation=1
+max-parallelism=180 segment-costs=[1764974287, 3667] cpu-comparison-result=287 [max(152 (self) vs 287 (sum children))]
12:AGGREGATE [STREAMING]
| output: sum(ss_ext_sales_price), sum(coalesce(sr_return_amt, CAST(0 AS DECIMAL(7,2)))), sum(ss_net_profit - coalesce(sr_net_loss, CAST(0 AS DECIMAL(7,2))))
| group by: s_store_id
| mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=6 row-size=76B cardinality=678 cost=369345185
+| tuple-ids=6 row-size=76B cardinality=678 cost=6485765
| in pipelines: 01(GETNEXT)
|
11:HASH JOIN [INNER JOIN, BROADCAST]
@@ -1806,16 +1812,16 @@ max-parallelism=390 segment-costs=[3880741788, 3667] cpu-comparison-result=287 [
| hash predicates: ss_store_sk = s_store_sk
| fk/pk conjuncts: ss_store_sk = s_store_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1N,4,5,2,3 row-size=117B cardinality=853.49M cost=373571195
+| tuple-ids=0,1N,2,4,5,3 row-size=117B cardinality=14.51M cost=6349072
| in pipelines: 01(GETNEXT), 04(OPEN)
|
|--F27:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.33MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | Per-Instance Resources: mem-estimate=23.33MB mem-reservation=23.25MB thread-reservation=1
| | max-parallelism=10 segment-costs=[3140]
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: s_store_sk
-| | runtime filters: RF000[bloom] <- s_store_sk, RF001[min_max] <- s_store_sk
+| | runtime filters: RF001[min_max] <- s_store_sk
| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=1350
| |
| 45:EXCHANGE [BROADCAST]
@@ -1838,65 +1844,27 @@ max-parallelism=390 segment-costs=[3880741788, 3667] cpu-comparison-result=287 [
|
10:HASH JOIN [INNER JOIN, BROADCAST]
| hash-table-id=01
-| hash predicates: ss_sold_date_sk = d_date_sk
-| fk/pk conjuncts: ss_sold_date_sk = d_date_sk
-| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1N,4,5,2 row-size=85B cardinality=853.49M cost=373571195
-| in pipelines: 01(GETNEXT), 03(OPEN)
-|
-|--F28:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
-| JOIN BUILD
-| | join-table-id=01 plan-id=02 cohort-id=01
-| | build expressions: d_date_sk
-| | runtime filters: RF002[bloom] <- d_date_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
-| |
-| 44:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=2 row-size=8B cardinality=7.30K cost=9700
-| | in pipelines: 03(GETNEXT)
-| |
-| F05:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
-| 03:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
-| HDFS partitions=1/1 files=1 size=2.17MB
-| predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| stored statistics:
-| table: rows=73.05K size=2.17MB
-| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=73.05K
-| parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
-| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
-| in pipelines: 03(GETNEXT)
-|
-09:HASH JOIN [INNER JOIN, BROADCAST]
-| hash-table-id=02
| hash predicates: ss_promo_sk = p_promo_sk
| fk/pk conjuncts: ss_promo_sk = p_promo_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1N,4,5 row-size=77B cardinality=853.49M cost=376155403
+| tuple-ids=0,1N,2,4,5 row-size=85B cardinality=14.51M cost=6392992
| in pipelines: 01(GETNEXT), 06(OPEN)
|
-|--F29:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=10
+|--F28:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=10
| | Per-Instance Resources: mem-estimate=24.30MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[4139]
| JOIN BUILD
-| | join-table-id=02 plan-id=03 cohort-id=01
+| | join-table-id=01 plan-id=02 cohort-id=01
| | build expressions: p_promo_sk
-| | runtime filters: RF004[bloom] <- p_promo_sk, RF005[min_max] <- p_promo_sk
+| | runtime filters: RF002[bloom] <- p_promo_sk, RF003[min_max] <- p_promo_sk
| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=1779
| |
-| 43:EXCHANGE [BROADCAST]
+| 44:EXCHANGE [BROADCAST]
| | mem-estimate=50.53KB mem-reservation=0B thread-reservation=0
| | tuple-ids=5 row-size=17B cardinality=1.78K cost=2360
| | in pipelines: 06(GETNEXT)
| |
-| F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| F05:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=16.08MB mem-reservation=32.00KB thread-reservation=1
| max-parallelism=1 segment-costs=[300]
| 06:SCAN HDFS [tpcds_partitioned_parquet_snap.promotion, RANDOM]
@@ -1912,29 +1880,29 @@ max-parallelism=390 segment-costs=[3880741788, 3667] cpu-comparison-result=287 [
| tuple-ids=5 row-size=17B cardinality=1.78K cost=180
| in pipelines: 06(GETNEXT)
|
-08:HASH JOIN [INNER JOIN, BROADCAST]
-| hash-table-id=03
+09:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=02
| hash predicates: ss_item_sk = i_item_sk
| fk/pk conjuncts: ss_item_sk = i_item_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
-| tuple-ids=0,1N,4 row-size=60B cardinality=863.56M(filtered from 863.56M) cost=377980972
+| tuple-ids=0,1N,2,4 row-size=68B cardinality=14.68M cost=40324146
| in pipelines: 01(GETNEXT), 05(OPEN)
|
-|--F30:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=10
+|--F29:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=10
| | Per-Instance Resources: mem-estimate=35.82MB mem-reservation=35.50MB thread-reservation=1 runtime-filters-memory=1.00MB
| | max-parallelism=10 segment-costs=[83840]
| JOIN BUILD
-| | join-table-id=03 plan-id=04 cohort-id=01
+| | join-table-id=02 plan-id=03 cohort-id=01
| | build expressions: i_item_sk
-| | runtime filters: RF006[bloom] <- i_item_sk, RF007[min_max] <- i_item_sk
+| | runtime filters: RF004[bloom] <- i_item_sk, RF005[min_max] <- i_item_sk
| | mem-estimate=34.50MB mem-reservation=34.50MB spill-buffer=128.00KB thread-reservation=0 cost=36000
| |
-| 42:EXCHANGE [BROADCAST]
+| 43:EXCHANGE [BROADCAST]
| | mem-estimate=329.25KB mem-reservation=0B thread-reservation=0
| | tuple-ids=4 row-size=8B cardinality=36.00K cost=47840
| | in pipelines: 05(GETNEXT)
| |
-| F03:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
+| F04:PLAN FRAGMENT [RANDOM] hosts=4 instances=4
| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=512.00KB thread-reservation=1
| max-parallelism=4 segment-costs=[53143]
| 05:SCAN HDFS [tpcds_partitioned_parquet_snap.item, RANDOM]
@@ -1950,17 +1918,55 @@ max-parallelism=390 segment-costs=[3880741788, 3667] cpu-comparison-result=287 [
| tuple-ids=4 row-size=8B cardinality=36.00K cost=51588
| in pipelines: 05(GETNEXT)
|
+08:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=03
+| hash predicates: ss_sold_date_sk = d_date_sk
+| fk/pk conjuncts: ss_sold_date_sk = d_date_sk
+| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,1N,2 row-size=60B cardinality=146.84M cost=64272307
+| in pipelines: 01(GETNEXT), 03(OPEN)
+|
+|--F30:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=10
+| | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[71]
+| JOIN BUILD
+| | join-table-id=03 plan-id=04 cohort-id=01
+| | build expressions: d_date_sk
+| | runtime filters: RF006[bloom] <- d_date_sk
+| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=31
+| |
+| 42:EXCHANGE [BROADCAST]
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=2 row-size=8B cardinality=31 cost=40
+| | in pipelines: 03(GETNEXT)
+| |
+| F03:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
+| max-parallelism=1 segment-costs=[12521]
+| 03:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
+| HDFS partitions=1/1 files=1 size=2.17MB
+| predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| stored statistics:
+| table: rows=73.05K size=2.17MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=73.05K
+| parquet statistics predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| parquet dictionary predicates: d_date <= DATE '1999-09-11', d_date >= DATE '1999-08-12'
+| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| tuple-ids=2 row-size=8B cardinality=31 cost=12520
+| in pipelines: 03(GETNEXT)
+|
07:HASH JOIN [LEFT OUTER JOIN, PARTITIONED]
| hash-table-id=04
| hash predicates: ss_item_sk = sr_item_sk, ss_ticket_number = sr_ticket_number
| fk/pk conjuncts: ss_item_sk = sr_item_sk, ss_ticket_number = sr_ticket_number
| mem-estimate=0B mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,1N row-size=52B cardinality=863.56M(filtered from 8.64G) cost=1787059842
+| tuple-ids=0,1N row-size=52B cardinality=146.84M(filtered from 8.64G) cost=1603220974
| in pipelines: 01(GETNEXT), 02(OPEN)
|
|--F31:PLAN FRAGMENT [HASH(ss_item_sk,ss_ticket_number)] hosts=10 instances=120
| | Per-Instance Resources: mem-estimate=354.94MB mem-reservation=34.00MB thread-reservation=1
-| | max-parallelism=390 segment-costs=[1039464723]
+| | max-parallelism=180 segment-costs=[1039464723]
| JOIN BUILD
| | join-table-id=04 plan-id=05 cohort-id=01
| | build expressions: sr_item_sk, sr_ticket_number
@@ -1977,7 +1983,7 @@ max-parallelism=390 segment-costs=[3880741788, 3667] cpu-comparison-result=287 [
| max-parallelism=190 segment-costs=[1806773027]
| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.store_returns, RANDOM]
| HDFS partitions=2004/2004 files=2004 size=33.63MB
-| runtime filters: RF007[min_max] -> tpcds_partitioned_parquet_snap.store_returns.sr_item_sk, RF006[bloom] -> tpcds_partitioned_parquet_snap.store_returns.sr_item_sk
+| runtime filters: RF005[min_max] -> tpcds_partitioned_parquet_snap.store_returns.sr_item_sk, RF004[bloom] -> tpcds_partitioned_parquet_snap.store_returns.sr_item_sk
| stored statistics:
| table: rows=863.99M size=48.14GB
| partitions: 2004/2004 rows=863.99M
@@ -1988,23 +1994,23 @@ max-parallelism=390 segment-costs=[3880741788, 3667] cpu-comparison-result=287 [
| in pipelines: 02(GETNEXT)
|
40:EXCHANGE [HASH(ss_item_sk,ss_ticket_number)]
-| mem-estimate=14.22MB mem-reservation=0B thread-reservation=0
-| tuple-ids=0 row-size=32B cardinality=863.56M(filtered from 8.64G) cost=223057996
+| mem-estimate=11.12MB mem-reservation=0B thread-reservation=0
+| tuple-ids=0 row-size=32B cardinality=146.84M(filtered from 8.64G) cost=37929031
| in pipelines: 01(GETNEXT)
|
-F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=120
-Per-Host Shared Resources: mem-estimate=4.00MB mem-reservation=4.00MB thread-reservation=0 runtime-filters-memory=4.00MB
+F00:PLAN FRAGMENT [RANDOM] hosts=10 instances=32 (adjusted from 120)
+Per-Host Shared Resources: mem-estimate=3.00MB mem-reservation=3.00MB thread-reservation=0 runtime-filters-memory=3.00MB
Per-Instance Resources: mem-estimate=32.88MB mem-reservation=8.00MB thread-reservation=1
-max-parallelism=630 segment-costs=[6205817173]
+max-parallelism=32 segment-costs=[445926760]
01:SCAN HDFS [tpcds_partitioned_parquet_snap.store_sales, RANDOM]
HDFS partitions=1824/1824 files=1824 size=199.33MB
- runtime filters: RF007[min_max] -> ss_item_sk, RF005[min_max] -> ss_promo_sk, RF001[min_max] -> ss_store_sk, RF006[bloom] -> ss_item_sk, RF004[bloom] -> ss_promo_sk, RF000[bloom] -> ss_store_sk, RF002[bloom] -> ss_sold_date_sk
+ runtime filters: RF005[min_max] -> ss_item_sk, RF003[min_max] -> ss_promo_sk, RF001[min_max] -> ss_store_sk, RF006[bloom] -> ss_sold_date_sk, RF004[bloom] -> ss_item_sk, RF002[bloom] -> ss_promo_sk
stored statistics:
table: rows=8.64G size=389.90GB
partitions: 1824/1824 rows=8.64G
columns: all
- extrapolated-rows=disabled max-scan-range-rows=390.22M
+ extrapolated-rows=disabled max-scan-range-rows=390.22M est-scan-range=32(filtered from 1824)
mem-estimate=16.00MB mem-reservation=8.00MB thread-reservation=0
- tuple-ids=0 row-size=32B cardinality=863.56M(filtered from 8.64G) cost=3981282136
+ tuple-ids=0 row-size=32B cardinality=146.84M(filtered from 8.64G) cost=67664334
in pipelines: 01(GETNEXT)
====
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q82.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q82.test
index 8a53e26ec..f824d71df 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q82.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q82.test
@@ -18,10 +18,10 @@ select i_item_id
# end query 82 in stream 0 using template query82.tpl
---- PLAN
Max Per-Host Resource Reservation: Memory=78.88MB Threads=1
-Per-Host Resource Estimates: Memory=125.61GB
+Per-Host Resource Estimates: Memory=34.02GB
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=125.61GB mem-reservation=78.88MB thread-reservation=1 runtime-filters-memory=3.00MB
-| max-parallelism=1 segment-costs=[22224911049, 58423014420, 300]
+| Per-Instance Resources: mem-estimate=34.02GB mem-reservation=78.88MB thread-reservation=1 runtime-filters-memory=3.00MB
+| max-parallelism=1 segment-costs=[6942365839, 12748523993, 300]
PLAN-ROOT SINK
| output exprs: i_item_id, i_item_desc, i_current_price
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0 cost=300
@@ -29,13 +29,13 @@ PLAN-ROOT SINK
08:TOP-N [LIMIT=100]
| order by: i_item_id ASC
| mem-estimate=14.12KB mem-reservation=0B thread-reservation=0
-| tuple-ids=5 row-size=145B cardinality=100 cost=58423014420
+| tuple-ids=5 row-size=145B cardinality=100 cost=12748523993
| in pipelines: 08(GETNEXT), 07(OPEN)
|
07:AGGREGATE [FINALIZE]
| group by: i_item_id, i_item_desc, i_current_price
-| mem-estimate=119.90GB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=4 row-size=145B cardinality=3.28G cost=19349708197
+| mem-estimate=28.31GB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=4 row-size=145B cardinality=766.16M cost=4522345595
| in pipelines: 07(GETNEXT), 03(OPEN)
|
06:HASH JOIN [INNER JOIN]
@@ -43,7 +43,7 @@ PLAN-ROOT SINK
| fk/pk conjuncts: inv_date_sk = d_date_sk
| runtime filters: RF000[bloom] <- d_date_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=3,0,1,2 row-size=177B cardinality=3.28G cost=1434860975
+| tuple-ids=3,0,1,2 row-size=177B cardinality=766.16M cost=979678367
| in pipelines: 03(GETNEXT), 02(OPEN)
|
|--02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
@@ -56,7 +56,7 @@ PLAN-ROOT SINK
| parquet statistics predicates: d_date <= DATE '2002-05-09', d_date >= DATE '2002-03-10'
| parquet dictionary predicates: d_date <= DATE '2002-05-09', d_date >= DATE '2002-03-10'
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
+| tuple-ids=2 row-size=8B cardinality=61 cost=12520
| in pipelines: 02(GETNEXT)
|
05:HASH JOIN [INNER JOIN]
@@ -117,7 +117,7 @@ PLAN-ROOT SINK
in pipelines: 03(GETNEXT)
---- DISTRIBUTEDPLAN
Max Per-Host Resource Reservation: Memory=1.28GB Threads=52
-Per-Host Resource Estimates: Memory=32.82GB
+Per-Host Resource Estimates: Memory=16.79GB
F06:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=5.70MB mem-reservation=4.00MB thread-reservation=1
| max-parallelism=1 segment-costs=[354] cpu-comparison-result=190 [max(1 (self) vs 190 (sum children))]
@@ -133,32 +133,32 @@ PLAN-ROOT SINK
| in pipelines: 08(GETNEXT)
|
F05:PLAN FRAGMENT [HASH(i_item_id,i_item_desc,i_current_price)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=1.40GB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=5850 segment-costs=[21894410797, 58423014420, 410] cpu-comparison-result=190 [max(120 (self) vs 190 (sum children))]
+Per-Instance Resources: mem-estimate=649.79MB mem-reservation=34.00MB thread-reservation=1
+max-parallelism=1280 segment-costs=[5117084517, 12748523993, 410] cpu-comparison-result=190 [max(120 (self) vs 190 (sum children))]
08:TOP-N [LIMIT=100]
| order by: i_item_id ASC
| mem-estimate=14.12KB mem-reservation=0B thread-reservation=0
-| tuple-ids=5 row-size=145B cardinality=100 cost=58423014420
+| tuple-ids=5 row-size=145B cardinality=100 cost=12748523993
| in pipelines: 08(GETNEXT), 14(OPEN)
|
14:AGGREGATE [FINALIZE]
| group by: i_item_id, i_item_desc, i_current_price
-| mem-estimate=1.37GB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=4 row-size=145B cardinality=3.28G cost=19349708197
+| mem-estimate=622.38MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=4 row-size=145B cardinality=766.16M cost=4522345595
| in pipelines: 14(GETNEXT), 03(OPEN)
|
13:EXCHANGE [HASH(i_item_id,i_item_desc,i_current_price)]
| mem-estimate=27.41MB mem-reservation=0B thread-reservation=0
-| tuple-ids=4 row-size=145B cardinality=3.28G cost=2544702600
+| tuple-ids=4 row-size=145B cardinality=766.16M cost=594738922
| in pipelines: 03(GETNEXT)
|
F03:PLAN FRAGMENT [HASH(i_item_sk)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=1.21GB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=3230 segment-costs=[21391408432, 32215887147] cpu-comparison-result=190 [max(190 (self) vs 185 (sum children))]
+Per-Instance Resources: mem-estimate=653.69MB mem-reservation=34.00MB thread-reservation=1
+max-parallelism=760 segment-costs=[6108870466, 7529383589] cpu-comparison-result=190 [max(190 (self) vs 185 (sum children))]
07:AGGREGATE [STREAMING]
| group by: i_item_id, i_item_desc, i_current_price
-| mem-estimate=1.12GB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=4 row-size=145B cardinality=3.28G cost=19349708197
+| mem-estimate=562.78MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=4 row-size=145B cardinality=766.16M cost=4522345595
| in pipelines: 03(GETNEXT)
|
06:HASH JOIN [INNER JOIN, BROADCAST]
@@ -166,26 +166,26 @@ max-parallelism=3230 segment-costs=[21391408432, 32215887147] cpu-comparison-res
| hash predicates: inv_date_sk = d_date_sk
| fk/pk conjuncts: inv_date_sk = d_date_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=3,0,1,2 row-size=177B cardinality=3.28G cost=1434853670
+| tuple-ids=3,0,1,2 row-size=177B cardinality=766.16M cost=979678306
| in pipelines: 03(GETNEXT), 02(OPEN)
|
|--F07:PLAN FRAGMENT [HASH(i_item_sk)] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
+| | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[141]
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: d_date_sk
| | runtime filters: RF000[bloom] <- d_date_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=61
| |
| 12:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=2 row-size=8B cardinality=7.30K cost=9700
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=2 row-size=8B cardinality=61 cost=80
| | in pipelines: 02(GETNEXT)
| |
| F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
+| max-parallelism=1 segment-costs=[12522]
| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| HDFS partitions=1/1 files=1 size=2.17MB
| predicates: d_date <= DATE '2002-05-09', d_date >= DATE '2002-03-10'
@@ -196,7 +196,7 @@ max-parallelism=3230 segment-costs=[21391408432, 32215887147] cpu-comparison-res
| parquet statistics predicates: d_date <= DATE '2002-05-09', d_date >= DATE '2002-03-10'
| parquet dictionary predicates: d_date <= DATE '2002-05-09', d_date >= DATE '2002-03-10'
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
+| tuple-ids=2 row-size=8B cardinality=61 cost=12520
| in pipelines: 02(GETNEXT)
|
05:HASH JOIN [INNER JOIN, PARTITIONED]
@@ -209,7 +209,7 @@ max-parallelism=3230 segment-costs=[21391408432, 32215887147] cpu-comparison-res
|
|--F08:PLAN FRAGMENT [HASH(i_item_sk)] hosts=10 instances=120
| | Per-Instance Resources: mem-estimate=60.18MB mem-reservation=35.00MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=3230 segment-costs=[120544102]
+| | max-parallelism=760 segment-costs=[120544102]
| JOIN BUILD
| | join-table-id=01 plan-id=02 cohort-id=01
| | build expressions: inv_item_sk
@@ -302,7 +302,7 @@ max-parallelism=70 segment-costs=[629576044]
in pipelines: 03(GETNEXT)
---- PARALLELPLANS
Max Per-Host Resource Reservation: Memory=1.28GB Threads=52
-Per-Host Resource Estimates: Memory=32.82GB
+Per-Host Resource Estimates: Memory=16.79GB
F06:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=5.70MB mem-reservation=4.00MB thread-reservation=1
| max-parallelism=1 segment-costs=[354] cpu-comparison-result=190 [max(1 (self) vs 190 (sum children))]
@@ -318,32 +318,32 @@ PLAN-ROOT SINK
| in pipelines: 08(GETNEXT)
|
F05:PLAN FRAGMENT [HASH(i_item_id,i_item_desc,i_current_price)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=1.40GB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=5850 segment-costs=[21894410797, 58423014420, 410] cpu-comparison-result=190 [max(120 (self) vs 190 (sum children))]
+Per-Instance Resources: mem-estimate=649.79MB mem-reservation=34.00MB thread-reservation=1
+max-parallelism=1280 segment-costs=[5117084517, 12748523993, 410] cpu-comparison-result=190 [max(120 (self) vs 190 (sum children))]
08:TOP-N [LIMIT=100]
| order by: i_item_id ASC
| mem-estimate=14.12KB mem-reservation=0B thread-reservation=0
-| tuple-ids=5 row-size=145B cardinality=100 cost=58423014420
+| tuple-ids=5 row-size=145B cardinality=100 cost=12748523993
| in pipelines: 08(GETNEXT), 14(OPEN)
|
14:AGGREGATE [FINALIZE]
| group by: i_item_id, i_item_desc, i_current_price
-| mem-estimate=1.37GB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=4 row-size=145B cardinality=3.28G cost=19349708197
+| mem-estimate=622.38MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=4 row-size=145B cardinality=766.16M cost=4522345595
| in pipelines: 14(GETNEXT), 03(OPEN)
|
13:EXCHANGE [HASH(i_item_id,i_item_desc,i_current_price)]
| mem-estimate=27.41MB mem-reservation=0B thread-reservation=0
-| tuple-ids=4 row-size=145B cardinality=3.28G cost=2544702600
+| tuple-ids=4 row-size=145B cardinality=766.16M cost=594738922
| in pipelines: 03(GETNEXT)
|
F03:PLAN FRAGMENT [HASH(i_item_sk)] hosts=10 instances=120
-Per-Instance Resources: mem-estimate=1.21GB mem-reservation=34.00MB thread-reservation=1
-max-parallelism=3230 segment-costs=[21391408432, 32215887147] cpu-comparison-result=190 [max(190 (self) vs 185 (sum children))]
+Per-Instance Resources: mem-estimate=653.69MB mem-reservation=34.00MB thread-reservation=1
+max-parallelism=760 segment-costs=[6108870466, 7529383589] cpu-comparison-result=190 [max(190 (self) vs 185 (sum children))]
07:AGGREGATE [STREAMING]
| group by: i_item_id, i_item_desc, i_current_price
-| mem-estimate=1.12GB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=4 row-size=145B cardinality=3.28G cost=19349708197
+| mem-estimate=562.78MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=4 row-size=145B cardinality=766.16M cost=4522345595
| in pipelines: 03(GETNEXT)
|
06:HASH JOIN [INNER JOIN, BROADCAST]
@@ -351,26 +351,26 @@ max-parallelism=3230 segment-costs=[21391408432, 32215887147] cpu-comparison-res
| hash predicates: inv_date_sk = d_date_sk
| fk/pk conjuncts: inv_date_sk = d_date_sk
| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=3,0,1,2 row-size=177B cardinality=3.28G cost=1434853670
+| tuple-ids=3,0,1,2 row-size=177B cardinality=766.16M cost=979678306
| in pipelines: 03(GETNEXT), 02(OPEN)
|
|--F07:PLAN FRAGMENT [HASH(i_item_sk)] hosts=10 instances=10
-| | Per-Instance Resources: mem-estimate=24.32MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=10 segment-costs=[17005]
+| | Per-Instance Resources: mem-estimate=24.27MB mem-reservation=24.25MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | max-parallelism=10 segment-costs=[141]
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: d_date_sk
| | runtime filters: RF000[bloom] <- d_date_sk
-| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=7305
+| | mem-estimate=23.25MB mem-reservation=23.25MB spill-buffer=64.00KB thread-reservation=0 cost=61
| |
| 12:EXCHANGE [BROADCAST]
-| | mem-estimate=69.07KB mem-reservation=0B thread-reservation=0
-| | tuple-ids=2 row-size=8B cardinality=7.30K cost=9700
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=2 row-size=8B cardinality=61 cost=80
| | in pipelines: 02(GETNEXT)
| |
| F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
| Per-Instance Resources: mem-estimate=16.05MB mem-reservation=1.00MB thread-reservation=1
-| max-parallelism=1 segment-costs=[12835]
+| max-parallelism=1 segment-costs=[12522]
| 02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim, RANDOM]
| HDFS partitions=1/1 files=1 size=2.17MB
| predicates: d_date <= DATE '2002-05-09', d_date >= DATE '2002-03-10'
@@ -381,7 +381,7 @@ max-parallelism=3230 segment-costs=[21391408432, 32215887147] cpu-comparison-res
| parquet statistics predicates: d_date <= DATE '2002-05-09', d_date >= DATE '2002-03-10'
| parquet dictionary predicates: d_date <= DATE '2002-05-09', d_date >= DATE '2002-03-10'
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
+| tuple-ids=2 row-size=8B cardinality=61 cost=12520
| in pipelines: 02(GETNEXT)
|
05:HASH JOIN [INNER JOIN, PARTITIONED]
@@ -394,7 +394,7 @@ max-parallelism=3230 segment-costs=[21391408432, 32215887147] cpu-comparison-res
|
|--F08:PLAN FRAGMENT [HASH(i_item_sk)] hosts=10 instances=120
| | Per-Instance Resources: mem-estimate=60.18MB mem-reservation=35.00MB thread-reservation=1 runtime-filters-memory=1.00MB
-| | max-parallelism=3230 segment-costs=[120544102]
+| | max-parallelism=760 segment-costs=[120544102]
| JOIN BUILD
| | join-table-id=01 plan-id=02 cohort-id=01
| | build expressions: inv_item_sk
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q92.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q92.test
index 6d3dbcbf9..8b8191519 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q92.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds_cpu_cost/tpcds-q92.test
@@ -30,11 +30,11 @@ limit 100;
# end query 92 in stream 0 using template query92.tpl
---- PLAN
-Max Per-Host Resource Reservation: Memory=60.50MB Threads=1
-Per-Host Resource Estimates: Memory=107MB
+Max Per-Host Resource Reservation: Memory=35.56MB Threads=1
+Per-Host Resource Estimates: Memory=91MB
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
-| Per-Instance Resources: mem-estimate=106.88MB mem-reservation=60.50MB thread-reservation=1 runtime-filters-memory=4.00MB
-| max-parallelism=1 segment-costs=[1951409959, 376784805, 0, 0]
+| Per-Instance Resources: mem-estimate=90.81MB mem-reservation=35.56MB thread-reservation=1 runtime-filters-memory=4.00MB
+| max-parallelism=1 segment-costs=[98260359, 19403912, 0, 0]
PLAN-ROOT SINK
| output exprs: sum(ws_ext_discount_amt)
| mem-estimate=0B mem-reservation=0B thread-reservation=0 cost=0
@@ -48,33 +48,26 @@ PLAN-ROOT SINK
10:AGGREGATE [FINALIZE]
| output: sum(ws_ext_discount_amt)
| mem-estimate=16.00KB mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=8 row-size=16B cardinality=1 cost=157419
-| in pipelines: 10(GETNEXT), 00(OPEN)
+| tuple-ids=8 row-size=16B cardinality=1 cost=7853
+| in pipelines: 10(GETNEXT), 06(OPEN)
|
-09:HASH JOIN [LEFT SEMI JOIN]
-| hash predicates: i_item_sk = ws_item_sk
+09:HASH JOIN [RIGHT SEMI JOIN]
+| hash predicates: ws_item_sk = i_item_sk
| other join predicates: ws_ext_discount_amt > CAST(1.3 AS DECIMAL(2,1)) * avg(ws_ext_discount_amt)
-| runtime filters: RF000[bloom] <- ws_item_sk, RF001[min_max] <- ws_item_sk
-| mem-estimate=34.00MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
-| tuple-ids=0,1,2 row-size=28B cardinality=2.25M cost=1344498
-| in pipelines: 00(GETNEXT), 06(OPEN)
-|
-|--06:AGGREGATE [FINALIZE]
-| | output: avg(ws_ext_discount_amt)
-| | group by: ws_item_sk
-| | mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB thread-reservation=0
-| | tuple-ids=6 row-size=12B cardinality=360.18K cost=632729691
-| | in pipelines: 06(GETNEXT), 03(OPEN)
-| |
-| 05:HASH JOIN [INNER JOIN]
+| runtime filters: RF000[bloom] <- i_item_sk, RF001[min_max] <- i_item_sk
+| mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
+| tuple-ids=0,1,2 row-size=28B cardinality=112.20K cost=224912
+| in pipelines: 06(GETNEXT), 00(OPEN)
+|
+|--08:HASH JOIN [INNER JOIN]
| | hash predicates: ws_sold_date_sk = d_date_sk
| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| | runtime filters: RF006[bloom] <- d_date_sk
+| | runtime filters: RF004[bloom] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| | tuple-ids=3,4 row-size=20B cardinality=2.16G cost=945425284
-| | in pipelines: 03(GETNEXT), 04(OPEN)
+| | tuple-ids=0,1,2 row-size=28B cardinality=112.20K cost=49199
+| | in pipelines: 00(GETNEXT), 02(OPEN)
| |
-| |--04:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
+| |--02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
| | HDFS partitions=1/1 files=1 size=2.17MB
| | predicates: d_date <= DATE '2001-06-11', d_date >= DATE '2001-03-13'
| | stored statistics:
@@ -84,30 +77,58 @@ PLAN-ROOT SINK
| | parquet statistics predicates: d_date <= DATE '2001-06-11', d_date >= DATE '2001-03-13'
| | parquet dictionary predicates: d_date <= DATE '2001-06-11', d_date >= DATE '2001-03-13'
| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| | tuple-ids=4 row-size=8B cardinality=7.30K cost=12520
-| | in pipelines: 04(GETNEXT)
+| | tuple-ids=2 row-size=8B cardinality=91 cost=12520
+| | in pipelines: 02(GETNEXT)
+| |
+| 07:HASH JOIN [INNER JOIN]
+| | hash predicates: ws_item_sk = i_item_sk
+| | fk/pk conjuncts: ws_item_sk = i_item_sk
+| | runtime filters: RF006[bloom] <- i_item_sk, RF007[min_max] <- i_item_sk
+| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=0,1 row-size=20B cardinality=112.20K(filtered from 2.25M) cost=436643
+| | in pipelines: 00(GETNEXT), 01(OPEN)
| |
-| 03:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales]
+| |--01:SCAN HDFS [tpcds_partitioned_parquet_snap.item]
+| | HDFS partitions=1/1 files=1 size=1.73MB
+| | predicates: i_manufact_id = CAST(561 AS INT)
+| | stored statistics:
+| | table: rows=360.00K size=33.54MB
+| | columns: all
+| | extrapolated-rows=disabled max-scan-range-rows=101.54K
+| | parquet statistics predicates: i_manufact_id = CAST(561 AS INT)
+| | parquet dictionary predicates: i_manufact_id = CAST(561 AS INT)
+| | mem-estimate=16.00MB mem-reservation=512.00KB thread-reservation=0
+| | tuple-ids=1 row-size=8B cardinality=375 cost=51588
+| | in pipelines: 01(GETNEXT)
+| |
+| 00:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales]
| HDFS partitions=1824/1824 files=1824 size=84.18MB
-| runtime filters: RF006[bloom] -> ws_sold_date_sk
+| runtime filters: RF007[min_max] -> ws_item_sk, RF004[bloom] -> ws_sold_date_sk, RF006[bloom] -> ws_item_sk
| stored statistics:
| table: rows=2.16G size=145.75GB
| partitions: 1824/1824 rows=2.16G
| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=3.37M
+| extrapolated-rows=disabled max-scan-range-rows=3.37M est-scan-range=91(filtered from 1824)
| mem-estimate=16.00MB mem-reservation=128.00KB thread-reservation=0
-| tuple-ids=3 row-size=12B cardinality=2.16G cost=373242464
-| in pipelines: 03(GETNEXT)
+| tuple-ids=0 row-size=12B cardinality=112.20K(filtered from 2.16G) cost=18621197
+| in pipelines: 00(GETNEXT)
|
-08:HASH JOIN [INNER JOIN]
+06:AGGREGATE [FINALIZE]
+| output: avg(ws_ext_discount_amt)
+| group by: ws_item_sk
+| mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB thread-reservation=0
+| tuple-ids=6 row-size=12B cardinality=360.18K cost=32459317
+| in pipelines: 06(GETNEXT), 03(OPEN)
+|
+05:HASH JOIN [INNER JOIN]
| hash predicates: ws_sold_date_sk = d_date_sk
| fk/pk conjuncts: ws_sold_date_sk = d_date_sk
| runtime filters: RF002[bloom] <- d_date_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1,2 row-size=28B cardinality=2.25M cost=991623
-| in pipelines: 00(GETNEXT), 02(OPEN)
+| tuple-ids=3,4 row-size=20B cardinality=107.76M cost=47167325
+| in pipelines: 03(GETNEXT), 04(OPEN)
|
-|--02:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
+|--04:SCAN HDFS [tpcds_partitioned_parquet_snap.date_dim]
| HDFS partitions=1/1 files=1 size=2.17MB
| predicates: d_date <= DATE '2001-06-11', d_date >= DATE '2001-03-13'
| stored statistics:
@@ -117,48 +138,26 @@ PLAN-ROOT SINK
| parquet statistics predicates: d_date <= DATE '2001-06-11', d_date >= DATE '2001-03-13'
| parquet dictionary predicates: d_date <= DATE '2001-06-11', d_date >= DATE '2001-03-13'
| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
-| tuple-ids=2 row-size=8B cardinality=7.30K cost=12520
-| in pipelines: 02(GETNEXT)
+| tuple-ids=4 row-size=8B cardinality=91 cost=12520
+| in pipelines: 04(GETNEXT)
|
-07:HASH JOIN [INNER JOIN]
-| hash predicates: ws_item_sk = i_item_sk
-| fk/pk conjuncts: ws_item_sk = i_item_sk
-| runtime filters: RF004[bloom] <- i_item_sk, RF005[min_max] <- i_item_sk
-| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
-| tuple-ids=0,1 row-size=20B cardinality=2.25M cost=984693
-| in pipelines: 00(GETNEXT), 01(OPEN)
-|
-|--01:SCAN HDFS [tpcds_partitioned_parquet_snap.item]
-| HDFS partitions=1/1 files=1 size=1.73MB
-| predicates: i_manufact_id = CAST(561 AS INT)
-| runtime filters: RF001[min_max] -> i_item_sk, RF000[bloom] -> i_item_sk
-| stored statistics:
-| table: rows=360.00K size=33.54MB
-| columns: all
-| extrapolated-rows=disabled max-scan-range-rows=101.54K
-| parquet statistics predicates: i_manufact_id = CAST(561 AS INT)
-| parquet dictionary predicates: i_manufact_id = CAST(561 AS INT)
-| mem-estimate=16.00MB mem-reservation=512.00KB thread-reservation=0
-| tuple-ids=1 row-size=8B cardinality=375 cost=51588
-| in pipelines: 01(GETNEXT)
-|
-00:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales]
+03:SCAN HDFS [tpcds_partitioned_parquet_snap.web_sales]
HDFS partitions=1824/1824 files=1824 size=84.18MB
- runtime filters: RF001[min_max] -> tpcds_partitioned_parquet_snap.web_sales.ws_item_sk, RF005[min_max] -> ws_item_sk, RF000[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_item_sk, RF002[bloom] -> ws_sold_date_sk, RF004[bloom] -> ws_item_sk
+ runtime filters: RF001[min_max] -> tpcds_partitioned_parquet_snap.web_sales.ws_item_sk, RF000[bloom] -> tpcds_partitioned_parquet_snap.web_sales.ws_item_sk, RF002[bloom] -> ws_sold_date_sk
... 2513 lines suppressed ...