You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by jc...@apache.org on 2020/08/10 14:18:16 UTC
[hive] branch master updated: HIVE-21196: Support semijoin
reduction on multiple column join (Stamatis Zampetakis,
reviewed by Jesus Camacho Rodriguez)
This is an automated email from the ASF dual-hosted git repository.
jcamacho pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new e6dd554 HIVE-21196: Support semijoin reduction on multiple column join (Stamatis Zampetakis, reviewed by Jesus Camacho Rodriguez)
e6dd554 is described below
commit e6dd5546d8224d0afe4b76c3e939781db183a897
Author: Stamatis Zampetakis <za...@gmail.com>
AuthorDate: Mon Aug 10 17:18:07 2020 +0300
HIVE-21196: Support semijoin reduction on multiple column join (Stamatis Zampetakis, reviewed by Jesus Camacho Rodriguez)
Closes apache/hive#1325
---
.../java/org/apache/hadoop/hive/conf/HiveConf.java | 5 +
.../apache/hadoop/hive/ql/exec/FunctionUtils.java | 28 +-
.../apache/hadoop/hive/ql/exec/OperatorUtils.java | 29 +
.../DynamicPartitionPruningOptimization.java | 3 +-
.../hive/ql/optimizer/SemiJoinReductionMerge.java | 501 +++++++++++++
.../apache/hadoop/hive/ql/parse/ParseContext.java | 14 +-
.../hadoop/hive/ql/parse/RuntimeValuesInfo.java | 14 +-
.../hadoop/hive/ql/parse/SemanticAnalyzer.java | 3 +-
.../apache/hadoop/hive/ql/parse/TezCompiler.java | 156 ++--
.../hadoop/hive/ql/plan/ExprNodeDescUtils.java | 31 +-
.../dynamic_semijoin_reduction_multicol.q | 94 +++
.../vectorized_dynamic_semijoin_reduction.q | 2 +
.../results/clientpositive/llap/acid_stats3.q.out | 6 +-
.../results/clientpositive/llap/acid_stats4.q.out | 10 +-
.../results/clientpositive/llap/acid_stats5.q.out | 10 +-
.../clientpositive/llap/allcolref_in_udf.q.out | 4 +-
.../clientpositive/llap/alterColumnStatsPart.q.out | 8 +-
.../clientpositive/llap/ambiguous_col.q.out | 12 +-
.../clientpositive/llap/analyze_tbl_date.q.out | 2 +-
.../llap/annotate_stats_deep_filters.q.out | 4 +-
.../llap/dynamic_semijoin_reduction.q.out | 28 +-
.../llap/dynamic_semijoin_reduction_2.q.out | 192 ++---
.../llap/dynamic_semijoin_reduction_multicol.q.out | 808 +++++++++++++++++++++
.../llap/dynamic_semijoin_user_level.q.out | 20 +-
.../clientpositive/llap/explainuser_1.q.out | 16 +-
.../clientpositive/llap/explainuser_2.q.out | 174 ++---
.../llap/join_reordering_no_stats.q.out | 4 +-
.../results/clientpositive/llap/ppd_join5.q.out | 6 +-
.../results/clientpositive/llap/semijoin.q.out | 4 +-
.../results/clientpositive/llap/semijoin4.q.out | 2 +-
.../results/clientpositive/llap/subquery_in.q.out | 12 +-
.../clientpositive/llap/subquery_multi.q.out | 4 +-
.../clientpositive/llap/subquery_multiinsert.q.out | 8 +-
.../clientpositive/llap/subquery_scalar.q.out | 4 +-
.../clientpositive/llap/subquery_select.q.out | 16 +-
.../llap/tez_fixed_bucket_pruning.q.out | 8 +-
.../vectorized_dynamic_semijoin_reduction.q.out | 95 +--
.../clientpositive/llap/vectorized_mapjoin3.q.out | 4 +-
.../clientpositive/perf/tez/cbo_query14.q.out | 6 +-
.../clientpositive/perf/tez/cbo_query24.q.out | 2 +-
.../clientpositive/perf/tez/cbo_query54.q.out | 8 +-
.../perf/tez/constraints/cbo_query14.q.out | 6 +-
.../perf/tez/constraints/cbo_query24.q.out | 2 +-
.../perf/tez/constraints/cbo_query54.q.out | 8 +-
.../perf/tez/constraints/query14.q.out | 662 ++++++++---------
.../perf/tez/constraints/query17.q.out | 104 +--
.../perf/tez/constraints/query24.q.out | 286 ++++----
.../perf/tez/constraints/query25.q.out | 102 +--
.../perf/tez/constraints/query29.q.out | 102 +--
.../perf/tez/constraints/query39.q.out | 68 +-
.../perf/tez/constraints/query40.q.out | 64 +-
.../perf/tez/constraints/query49.q.out | 206 +++---
.../perf/tez/constraints/query5.q.out | 217 +++---
.../perf/tez/constraints/query50.q.out | 64 +-
.../perf/tez/constraints/query54.q.out | 216 +++---
.../perf/tez/constraints/query64.q.out | 362 ++++-----
.../perf/tez/constraints/query72.q.out | 114 +--
.../perf/tez/constraints/query75.q.out | 334 ++++-----
.../perf/tez/constraints/query78.q.out | 197 ++---
.../perf/tez/constraints/query80.q.out | 222 +++---
.../perf/tez/constraints/query85.q.out | 88 +--
.../perf/tez/constraints/query93.q.out | 58 +-
.../results/clientpositive/perf/tez/query14.q.out | 632 ++++++++--------
.../results/clientpositive/perf/tez/query17.q.out | 108 +--
.../results/clientpositive/perf/tez/query24.q.out | 144 ++--
.../results/clientpositive/perf/tez/query25.q.out | 106 +--
.../results/clientpositive/perf/tez/query29.q.out | 106 +--
.../results/clientpositive/perf/tez/query39.q.out | 94 +--
.../results/clientpositive/perf/tez/query40.q.out | 68 +-
.../results/clientpositive/perf/tez/query49.q.out | 206 +++---
.../results/clientpositive/perf/tez/query5.q.out | 223 +++---
.../results/clientpositive/perf/tez/query50.q.out | 76 +-
.../results/clientpositive/perf/tez/query54.q.out | 218 +++---
.../results/clientpositive/perf/tez/query59.q.out | 86 +--
.../results/clientpositive/perf/tez/query64.q.out | 340 ++++-----
.../results/clientpositive/perf/tez/query72.q.out | 128 ++--
.../results/clientpositive/perf/tez/query75.q.out | 340 ++++-----
.../results/clientpositive/perf/tez/query78.q.out | 207 +++---
.../results/clientpositive/perf/tez/query80.q.out | 234 +++---
.../results/clientpositive/perf/tez/query85.q.out | 100 +--
.../results/clientpositive/perf/tez/query93.q.out | 58 +-
81 files changed, 5359 insertions(+), 3954 deletions(-)
diff --git a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
index 742dddf..2e7247b 100644
--- a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
+++ b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
@@ -4338,6 +4338,11 @@ public class HiveConf extends Configuration {
"Big table for runtime filteting should be of atleast this size"),
TEZ_DYNAMIC_SEMIJOIN_REDUCTION_THRESHOLD("hive.tez.dynamic.semijoin.reduction.threshold", (float) 0.50,
"Only perform semijoin optimization if the estimated benefit at or above this fraction of the target table"),
+ TEZ_DYNAMIC_SEMIJOIN_REDUCTION_MULTICOLUMN(
+ "hive.tez.dynamic.semijoin.reduction.multicolumn",
+ true,
+ "Whether to consider multicolumn semijoin reducers or not.\n"
+ + "This should always be set to true. Since it is a new feature, it has been made configurable."),
TEZ_DYNAMIC_SEMIJOIN_REDUCTION_FOR_MAPJOIN("hive.tez.dynamic.semijoin.reduction.for.mapjoin", false,
"Use a semi-join branch for map-joins. This may not make it faster, but is helpful in certain join patterns."),
TEZ_DYNAMIC_SEMIJOIN_REDUCTION_FOR_DPP_FACTOR("hive.tez.dynamic.semijoin.reduction.for.dpp.factor",
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionUtils.java b/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionUtils.java
index d63d317..c86954c 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionUtils.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionUtils.java
@@ -18,20 +18,24 @@
package org.apache.hadoop.hive.ql.exec;
-import java.util.List;
-
+import com.google.common.collect.HashMultimap;
+import com.google.common.collect.Multimap;
import org.apache.hadoop.hive.metastore.api.ResourceType;
import org.apache.hadoop.hive.metastore.api.ResourceUri;
import org.apache.hadoop.hive.ql.exec.FunctionInfo.FunctionResource;
import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.ql.plan.AggregationDesc;
import org.apache.hadoop.hive.ql.session.SessionState;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator;
import org.apache.hadoop.hive.ql.udf.generic.GenericUDAFResolver;
import org.apache.hadoop.hive.ql.udf.generic.GenericUDF;
import org.apache.hadoop.hive.ql.udf.generic.GenericUDTF;
import org.apache.hadoop.hive.ql.udf.ptf.TableFunctionResolver;
-import com.google.common.collect.HashMultimap;
-import com.google.common.collect.Multimap;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+
/**
* Function related utilities.
@@ -41,6 +45,22 @@ public final class FunctionUtils {
throw new UnsupportedOperationException("FunctionUtils should not be instantiated");
}
+ /**
+ * Extracts the UDAFE evaluators of the specified class from the provided aggregations.
+ */
+ public static <T extends GenericUDAFEvaluator> List<T> extractEvaluators(
+ Collection<? extends AggregationDesc> aggregations, Class<T> clazz) {
+ List<T> result = new ArrayList<>();
+ for (AggregationDesc d : aggregations) {
+ if (clazz.isInstance(d.getGenericUDAFEvaluator())) {
+ @SuppressWarnings("unchecked")
+ T t = (T) d.getGenericUDAFEvaluator();
+ result.add(t);
+ }
+ }
+ return result;
+ }
+
public static FunctionResource[] toFunctionResource(List<ResourceUri> resources) throws HiveException {
if (resources == null) {
return null;
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java b/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java
index 435cd92..553be93 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java
@@ -53,6 +53,35 @@ public class OperatorUtils {
private static final Logger LOG = LoggerFactory.getLogger(OperatorUtils.class);
+ /**
+ * Return the ancestor of the specified operator at the provided path or null if the path is invalid.
+ *
+ * The method is equivalent to following code:
+ * <pre>{@code
+ * op.getParentOperators().get(path[0])
+ * .getParentOperators().get(path[1])
+ * ...
+ * .getParentOperators().get(path[n])
+ * }</pre>
+ * with additional checks about the validity of the provided path and the type of the ancestor.
+ *
+ * @param op the operator for which we
+ * @param clazz the class of the ancestor operator
+ * @param path the path leading to the desired ancestor
+ * @param <T> the type of the ancestor
+ * @return the ancestor of the specified operator at the provided path or null if the path is invalid.
+ */
+ public static <T> T ancestor(Operator<?> op, Class<T> clazz, int... path) {
+ Operator<?> target = op;
+ for (int i = 0; i < path.length; i++) {
+ if (target.getParentOperators() == null || path[i] > target.getParentOperators().size()) {
+ return null;
+ }
+ target = target.getParentOperators().get(path[i]);
+ }
+ return clazz.isInstance(target) ? clazz.cast(target) : null;
+ }
+
public static <T> Set<T> findOperators(Operator<?> start, Class<T> clazz) {
return findOperators(start, clazz, new HashSet<T>());
}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
index 1dde261..e62bef6 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
@@ -20,6 +20,7 @@ package org.apache.hadoop.hive.ql.optimizer;
import java.util.ArrayList;
import java.util.Arrays;
+import java.util.Collections;
import java.util.EnumSet;
import java.util.HashMap;
import java.util.List;
@@ -847,7 +848,7 @@ public class DynamicPartitionPruningOptimization implements SemanticNodeProcesso
runtimeValuesInfo.setTableDesc(rsFinalTableDesc);
runtimeValuesInfo.setDynamicValueIDs(dynamicValueIDs);
runtimeValuesInfo.setColExprs(rsValueCols);
- runtimeValuesInfo.setTsColExpr(colExpr);
+ runtimeValuesInfo.setTargetColumns(Collections.singletonList(colExpr));
parseContext.getRsToRuntimeValuesInfoMap().put(rsOpFinal, runtimeValuesInfo);
parseContext.getColExprToGBMap().put(key, gb);
}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SemiJoinReductionMerge.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SemiJoinReductionMerge.java
new file mode 100644
index 0000000..f7e2c5c
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SemiJoinReductionMerge.java
@@ -0,0 +1,501 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer;
+
+import com.google.common.base.Preconditions;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.ql.exec.ColumnInfo;
+import org.apache.hadoop.hive.ql.exec.FilterOperator;
+import org.apache.hadoop.hive.ql.exec.FunctionUtils;
+import org.apache.hadoop.hive.ql.exec.GroupByOperator;
+import org.apache.hadoop.hive.ql.exec.Operator;
+import org.apache.hadoop.hive.ql.exec.OperatorFactory;
+import org.apache.hadoop.hive.ql.exec.OperatorUtils;
+import org.apache.hadoop.hive.ql.exec.ReduceSinkOperator;
+import org.apache.hadoop.hive.ql.exec.RowSchema;
+import org.apache.hadoop.hive.ql.exec.SelectOperator;
+import org.apache.hadoop.hive.ql.exec.TableScanOperator;
+import org.apache.hadoop.hive.ql.exec.Utilities;
+import org.apache.hadoop.hive.ql.io.AcidUtils;
+import org.apache.hadoop.hive.ql.parse.GenTezUtils;
+import org.apache.hadoop.hive.ql.parse.ParseContext;
+import org.apache.hadoop.hive.ql.parse.RuntimeValuesInfo;
+import org.apache.hadoop.hive.ql.parse.SemanticAnalyzer;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+import org.apache.hadoop.hive.ql.parse.SemiJoinBranchInfo;
+import org.apache.hadoop.hive.ql.plan.AggregationDesc;
+import org.apache.hadoop.hive.ql.plan.DynamicValue;
+import org.apache.hadoop.hive.ql.plan.ExprNodeColumnDesc;
+import org.apache.hadoop.hive.ql.plan.ExprNodeConstantDesc;
+import org.apache.hadoop.hive.ql.plan.ExprNodeDesc;
+import org.apache.hadoop.hive.ql.plan.ExprNodeDynamicValueDesc;
+import org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc;
+import org.apache.hadoop.hive.ql.plan.FilterDesc;
+import org.apache.hadoop.hive.ql.plan.GroupByDesc;
+import org.apache.hadoop.hive.ql.plan.PlanUtils;
+import org.apache.hadoop.hive.ql.plan.ReduceSinkDesc;
+import org.apache.hadoop.hive.ql.plan.SelectDesc;
+import org.apache.hadoop.hive.ql.plan.TableDesc;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDAFBloomFilter.GenericUDAFBloomFilterEvaluator;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMax;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMin;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFBetween;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFInBloomFilter;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFMurmurHash;
+import org.apache.hadoop.hive.ql.util.NullOrdering;
+import org.apache.hadoop.hive.serde2.typeinfo.TypeInfo;
+import org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory;
+
+import java.util.ArrayDeque;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Deque;
+import java.util.EnumSet;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+
+import static com.google.common.base.Preconditions.checkState;
+import static org.apache.hadoop.hive.ql.plan.ExprNodeDescUtils.and;
+
+/**
+ * A transformation that merges multiple single-column semi join reducers to one multi-column semi join reducer.
+ * <p>
+ * Semi join reducers are comprised from two parts:
+ * <ul>
+ * <li>Filter creation from the source relation and broadcast: SOURCE - SEL - GB1 - RS1 - GB2 - RS2</li>
+ * <li>Filter application on the target relation: TS - FIL[in_bloom(col)]</li>
+ * </ul>
+ * </p>
+ * <p>
+ * An example of the transformation on three single column semi join reducers is shown below. The plan is simplified for
+ * presentation purposes.
+ * <h3>BEFORE:</h3>
+ * <pre>
+ * / SEL[fname] - GB1 - RS1 - GB2 - RS2 \
+ * SOURCE - SEL[lname] - GB1 - RS1 - GB2 - RS2 -> TS[Author] - FIL[in_bloom(fname) ^ in_bloom(lname) ^ in_bloom(age)]
+ * \ SEL[age] - GB1 - RS1 - GB2 - RS2 /
+ * </pre>
+ * <h3>AFTER:</h3>
+ * <pre>
+ * SOURCE - SEL[fname, lname, age] - GB1 - RS1 - GB2 - RS2 -> TS[Author] - FIL[in_bloom(hash(fname,lname,age)]
+ * </pre>
+ * </p>
+ */
+public class SemiJoinReductionMerge extends Transform {
+
+ public ParseContext transform(ParseContext parseContext) throws SemanticException {
+ Map<ReduceSinkOperator, SemiJoinBranchInfo> allSemijoins = parseContext.getRsToSemiJoinBranchInfo();
+ if (allSemijoins.isEmpty()) {
+ return parseContext;
+ }
+ HiveConf hiveConf = parseContext.getConf();
+
+ for (Entry<SJSourceTarget, List<ReduceSinkOperator>> sjMergeCandidate : createMergeCandidates(allSemijoins)) {
+ final List<ReduceSinkOperator> sjBranches = sjMergeCandidate.getValue();
+ if (sjBranches.size() < 2) {
+ continue;
+ }
+ List<SelectOperator> selOps = new ArrayList<>(sjBranches.size());
+ for (ReduceSinkOperator rs : sjBranches) {
+ selOps.add(OperatorUtils.ancestor(rs, SelectOperator.class, 0, 0, 0, 0));
+ }
+
+ long sjEntriesHint = extractBloomEntriesHint(sjBranches);
+
+ SelectOperator selectOp = mergeSelectOps(sjMergeCandidate.getKey().source, selOps);
+
+ GroupByOperator gbPartialOp = createGroupBy(selectOp, selectOp, GroupByDesc.Mode.HASH, sjEntriesHint, hiveConf);
+
+ ReduceSinkOperator rsPartialOp = createReduceSink(gbPartialOp, NullOrdering.defaultNullOrder(hiveConf));
+ rsPartialOp.getConf().setReducerTraits(EnumSet.of(ReduceSinkDesc.ReducerTraits.QUICKSTART));
+
+ GroupByOperator gbCompleteOp =
+ createGroupBy(selectOp, rsPartialOp, GroupByDesc.Mode.FINAL, sjEntriesHint, hiveConf);
+
+ ReduceSinkOperator rsCompleteOp = createReduceSink(gbCompleteOp, NullOrdering.defaultNullOrder(hiveConf));
+
+ final TableScanOperator sjTargetTable = sjMergeCandidate.getKey().target;
+ SemiJoinBranchInfo sjInfo = new SemiJoinBranchInfo(sjTargetTable, false);
+ parseContext.getRsToSemiJoinBranchInfo().put(rsCompleteOp, sjInfo);
+
+ // Save the info that is required at query time to resolve dynamic/runtime values.
+ RuntimeValuesInfo valuesInfo = createRuntimeValuesInfo(rsCompleteOp, sjBranches, parseContext);
+ parseContext.getRsToRuntimeValuesInfoMap().put(rsCompleteOp, valuesInfo);
+
+ ExprNodeGenericFuncDesc sjPredicate = createSemiJoinPredicate(sjBranches, valuesInfo, parseContext);
+
+ // Update filter operators with the new semi-join predicate
+ for (Operator<?> op : sjTargetTable.getChildOperators()) {
+ if (op instanceof FilterOperator) {
+ FilterDesc filter = ((FilterOperator) op).getConf();
+ filter.setPredicate(and(filter.getPredicate(), sjPredicate));
+ }
+ }
+ // Update tableScan with the new semi-join predicate
+ sjTargetTable.getConf().setFilterExpr(and(sjTargetTable.getConf().getFilterExpr(), sjPredicate));
+
+ for (ReduceSinkOperator rs : sjBranches) {
+ GenTezUtils.removeSemiJoinOperator(parseContext, rs, sjTargetTable);
+ GenTezUtils.removeBranch(rs);
+ }
+ }
+ return parseContext;
+ }
+
+ /**
+ * Groups single column semijoin reducer branches together if they have the same source and target relation.
+ */
+ private static Collection<Map.Entry<SJSourceTarget, List<ReduceSinkOperator>>> createMergeCandidates(
+ Map<ReduceSinkOperator, SemiJoinBranchInfo> semijoins) {
+ // Predictable iteration helps to avoid small changes in the plans
+ LinkedHashMap<SJSourceTarget, List<ReduceSinkOperator>> sjGroups = new LinkedHashMap<>();
+ for (Map.Entry<ReduceSinkOperator, SemiJoinBranchInfo> smjEntry : semijoins.entrySet()) {
+ TableScanOperator ts = smjEntry.getValue().getTsOp();
+ // Semijoin optimization branch should look like <Parent>-SEL-GB1-RS1-GB2-RS2
+ SelectOperator selOp = OperatorUtils.ancestor(smjEntry.getKey(), SelectOperator.class, 0, 0, 0, 0);
+ Operator<?> source = selOp.getParentOperators().get(0);
+ checkState(selOp.getParentOperators().size() == 1, "Semijoin branches should not have multiple parents");
+ SJSourceTarget sjKey = new SJSourceTarget(source, ts);
+ List<ReduceSinkOperator> ops = sjGroups.computeIfAbsent(sjKey, tableScanOperator -> new ArrayList<>());
+ ops.add(smjEntry.getKey());
+ }
+ return sjGroups.entrySet();
+ }
+
+ /**
+ * Extracts a hint about the number of expected entries in the composite bloom filter from the hints in the specified
+ * (single column) semijoin branches.
+ * <p>
+ * If none of the individual semijoin branches has hints then the method returns -1, otherwise it returns the max.
+ * </p>
+ */
+ private static long extractBloomEntriesHint(List<ReduceSinkOperator> sjBranches) {
+ long bloomEntries = -1;
+ for (ReduceSinkOperator rs : sjBranches) {
+ GroupByOperator gbOp = OperatorUtils.ancestor(rs, GroupByOperator.class, 0, 0, 0);
+ List<GenericUDAFBloomFilterEvaluator> blooms =
+ FunctionUtils.extractEvaluators(gbOp.getConf().getAggregators(), GenericUDAFBloomFilterEvaluator.class);
+ Preconditions.checkState(blooms.size() == 1);
+ if (blooms.get(0).hasHintEntries()) {
+ bloomEntries = Math.max(bloomEntries, blooms.get(0).getExpectedEntries());
+ }
+ }
+ return bloomEntries;
+ }
+
+ /**
+ * Creates the multi-column semi-join predicate that is applied on the target relation.
+ *
+ * Assuming that the target columns of the semi-join are fname, lname, and age, the generated predicates is:
+ * <pre>
+ * fname BETWEEN ?min_fname AND ?max_fname and
+ * lname BETWEEN ?min_lname AND ?max_lname and
+ * age BETWEEN ?min_age AND ?max_age and
+ * IN_BLOOM_FILTER(HASH(fname,lname,age),?bloom_filter)
+ * </pre>
+ * where the question mark (?) indicates dynamic values bound at runtime.
+ */
+ private static ExprNodeGenericFuncDesc createSemiJoinPredicate(List<ReduceSinkOperator> sjBranches,
+ RuntimeValuesInfo sjValueInfo, ParseContext context) {
+ // Performance note: To speed-up evaluation 'BETWEEN' predicates should come before the 'IN_BLOOM_FILTER'
+ Deque<String> dynamicIds = new ArrayDeque<>(sjValueInfo.getDynamicValueIDs());
+ List<ExprNodeDesc> sjPredicates = new ArrayList<>();
+ List<ExprNodeDesc> hashArgs = new ArrayList<>();
+ for (ReduceSinkOperator rs : sjBranches) {
+ RuntimeValuesInfo info = context.getRsToRuntimeValuesInfoMap().get(rs);
+ checkState(info.getTargetColumns().size() == 1, "Cannot handle multi-column semijoin branches.");
+ final ExprNodeDesc targetColumn = info.getTargetColumns().get(0);
+ TypeInfo typeInfo = targetColumn.getTypeInfo();
+ DynamicValue minDynamic = new DynamicValue(dynamicIds.poll(), typeInfo);
+ DynamicValue maxDynamic = new DynamicValue(dynamicIds.poll(), typeInfo);
+
+ List<ExprNodeDesc> betweenArgs = Arrays.asList(
+ // Use false to not invert between result
+ new ExprNodeConstantDesc(Boolean.FALSE),
+ targetColumn,
+ new ExprNodeDynamicValueDesc(minDynamic),
+ new ExprNodeDynamicValueDesc(maxDynamic));
+ ExprNodeDesc betweenExp =
+ new ExprNodeGenericFuncDesc(TypeInfoFactory.booleanTypeInfo, new GenericUDFBetween(), "between", betweenArgs);
+ sjPredicates.add(betweenExp);
+ hashArgs.add(targetColumn);
+ }
+
+ ExprNodeDesc hashExp =
+ new ExprNodeGenericFuncDesc(TypeInfoFactory.intTypeInfo, new GenericUDFMurmurHash(), "hash", hashArgs);
+
+ assert dynamicIds.size() == 1 : "There should be one column left untreated the one with the bloom filter";
+ DynamicValue bloomDynamic = new DynamicValue(dynamicIds.poll(), TypeInfoFactory.binaryTypeInfo);
+ sjPredicates.add(
+ new ExprNodeGenericFuncDesc(TypeInfoFactory.booleanTypeInfo, new GenericUDFInBloomFilter(), "in_bloom_filter",
+ Arrays.asList(hashExp, new ExprNodeDynamicValueDesc(bloomDynamic))));
+ return and(sjPredicates);
+ }
+
+ /**
+ * Creates the necessary information in order to execute the multi-column semi join and perform further optimizations.
+ */
+ private static RuntimeValuesInfo createRuntimeValuesInfo(ReduceSinkOperator rs, List<ReduceSinkOperator> sjBranches,
+ ParseContext parseContext) {
+ List<ExprNodeDesc> valueCols = rs.getConf().getValueCols();
+ RuntimeValuesInfo info = new RuntimeValuesInfo();
+ TableDesc rsFinalTableDesc =
+ PlanUtils.getReduceValueTableDesc(PlanUtils.getFieldSchemasFromColumnList(valueCols, "_col"));
+ List<String> dynamicValueIDs = new ArrayList<>();
+ for (ExprNodeDesc rsCol : valueCols) {
+ dynamicValueIDs.add(rs.toString() + rsCol.getExprString());
+ }
+
+ info.setTableDesc(rsFinalTableDesc);
+ info.setDynamicValueIDs(dynamicValueIDs);
+ info.setColExprs(valueCols);
+ List<ExprNodeDesc> targetTableExpressions = new ArrayList<>();
+ for (ReduceSinkOperator sjBranch : sjBranches) {
+ RuntimeValuesInfo sjInfo = parseContext.getRsToRuntimeValuesInfoMap().get(sjBranch);
+ checkState(sjInfo.getTargetColumns().size() == 1, "Cannot handle multi-column semijoin branches.");
+ targetTableExpressions.add(sjInfo.getTargetColumns().get(0));
+ }
+ info.setTargetColumns(targetTableExpressions);
+ return info;
+ }
+
+ /**
+ * Merges multiple select operators in a single one appending an additional column that is the hash of all the others.
+ *
+ * <pre>
+ * Input: SEL[fname], SEL[lname], SEL[age]
+ * Output: SEL[fname, lname, age, hash(fname, lname, age)]
+ * </pre>
+ */
+ private static SelectOperator mergeSelectOps(Operator<?> parent, List<SelectOperator> selectOperators) {
+ List<String> colNames = new ArrayList<>();
+ List<ExprNodeDesc> colDescs = new ArrayList<>();
+ List<ColumnInfo> columnInfos = new ArrayList<>();
+ Map<String, ExprNodeDesc> selectColumnExprMap = new HashMap<>();
+ for (SelectOperator sel : selectOperators) {
+ checkState(sel.getConf().getColList().size() == 1);
+ ExprNodeDesc col = sel.getConf().getColList().get(0);
+ String colName = HiveConf.getColumnInternalName(colDescs.size());
+ colNames.add(colName);
+ columnInfos.add(new ColumnInfo(colName, col.getTypeInfo(), "", false));
+ colDescs.add(col);
+ selectColumnExprMap.put(colName, col);
+ }
+ ExprNodeDesc hashExp =
+ new ExprNodeGenericFuncDesc(TypeInfoFactory.intTypeInfo, new GenericUDFMurmurHash(), "hash", colDescs);
+ String hashName = HiveConf.getColumnInternalName(colDescs.size() + 1);
+ colNames.add(hashName);
+ columnInfos.add(new ColumnInfo(hashName, hashExp.getTypeInfo(), "", false));
+ // The n-1 columns in selDescs are used as parameters to min/max aggregations
+ List<ExprNodeDesc> selDescs = new ArrayList<>(colDescs);
+ // The nth column in selDescs is used as parameter to the bloom_filter aggregation
+ selDescs.add(hashExp);
+
+ SelectDesc select = new SelectDesc(selDescs, colNames);
+ SelectOperator selectOp =
+ (SelectOperator) OperatorFactory.getAndMakeChild(select, new RowSchema(columnInfos), parent);
+ selectOp.setColumnExprMap(selectColumnExprMap);
+ return selectOp;
+ }
+
+ /**
+ * Creates a reduce sink operator that emits all columns of the parent as values.
+ */
+ private static ReduceSinkOperator createReduceSink(Operator<?> parentOp, NullOrdering nullOrder)
+ throws SemanticException {
+ List<ExprNodeDesc> valueCols = new ArrayList<>();
+ RowSchema parentSchema = parentOp.getSchema();
+ List<String> outColNames = new ArrayList<>();
+ for (int i = 0; i < parentSchema.getSignature().size(); i++) {
+ ColumnInfo colInfo = parentSchema.getSignature().get(i);
+ ExprNodeColumnDesc colExpr = new ExprNodeColumnDesc(colInfo.getType(), colInfo.getInternalName(), "", false);
+ valueCols.add(colExpr);
+ outColNames.add(SemanticAnalyzer.getColumnInternalName(i));
+ }
+
+ ReduceSinkDesc rsDesc = PlanUtils
+ .getReduceSinkDesc(Collections.emptyList(), valueCols, outColNames, false, -1, 0, 1,
+ AcidUtils.Operation.NOT_ACID, nullOrder);
+ rsDesc.setColumnExprMap(Collections.emptyMap());
+ return (ReduceSinkOperator) OperatorFactory.getAndMakeChild(rsDesc, new RowSchema(parentSchema), parentOp);
+ }
+
+ /**
+ * Creates a group by operator with min, max, and bloomFilter aggregations for every column of the parent.
+ * <p>
+ * The method generates two kind of group by operators for intermediate and final aggregations respectively.
+ * Intermediate aggregations require the parent operator to be a select operator while final aggregations assume that
+ * the parent is a reduce sink operator.
+ * </p>
+ * <p>
+ * Intermediate group by example.
+ * <pre>
+ * Input: SEL[fname, lname, age, hash(fname,lname,age)]
+ * Output: GBY[min(fname),max(fname),min(lname),max(lname),min(age),max(age),bloom(hash)]
+ * </pre>
+ * </p>
+ * <p>
+ * Final group by example.
+ * <pre>
+ * Input: RS[fname_min,fname_max,lname_min,lname_max,age_min,age_max, hash_bloom]
+ * Output: GBY[min(fname_min),max(fname_max),min(lname_min),max(lname_max),min(age_min),max(age_max),bloom(hash_bloom)]
+ * </pre>
+ * </p>
+ */
+ private static GroupByOperator createGroupBy(SelectOperator selectOp, Operator<?> parentOp, GroupByDesc.Mode gbMode,
+ long bloomEntriesHint, HiveConf hiveConf) {
+
+ final List<ExprNodeDesc> params;
+ final GenericUDAFEvaluator.Mode udafMode = SemanticAnalyzer.groupByDescModeToUDAFMode(gbMode, false);
+ switch (gbMode) {
+ case FINAL:
+ params = createGroupByAggregationParameters((ReduceSinkOperator) parentOp);
+ break;
+ case HASH:
+ params = createGroupByAggregationParameters(selectOp);
+ break;
+ default:
+ throw new AssertionError(gbMode.toString() + " is not supported");
+ }
+
+ List<AggregationDesc> gbAggs = new ArrayList<>();
+ Deque<ExprNodeDesc> paramsCopy = new ArrayDeque<>(params);
+ while (paramsCopy.size() > 1) {
+ gbAggs.add(minAggregation(udafMode, paramsCopy.poll()));
+ gbAggs.add(maxAggregation(udafMode, paramsCopy.poll()));
+ }
+ gbAggs.add(bloomFilterAggregation(udafMode, paramsCopy.poll(), selectOp, bloomEntriesHint, hiveConf));
+ assert paramsCopy.size() == 0;
+
+ List<String> gbOutputNames = new ArrayList<>(gbAggs.size());
+ List<ColumnInfo> gbColInfos = new ArrayList<>(gbAggs.size());
+ for (int i = 0; i < params.size(); i++) {
+ String colName = HiveConf.getColumnInternalName(i);
+ gbOutputNames.add(colName);
+ final TypeInfo colType;
+ if (i == params.size() - 1) {
+ colType = TypeInfoFactory.binaryTypeInfo; // Bloom type
+ } else {
+ colType = params.get(i).getTypeInfo(); // Min/Max type
+ }
+ gbColInfos.add(new ColumnInfo(colName, colType, "", false));
+ }
+
+ float groupByMemoryUsage = HiveConf.getFloatVar(hiveConf, HiveConf.ConfVars.HIVEMAPAGGRHASHMEMORY);
+ float memoryThreshold = HiveConf.getFloatVar(hiveConf, HiveConf.ConfVars.HIVEMAPAGGRMEMORYTHRESHOLD);
+ float minReductionHashAggr = HiveConf.getFloatVar(hiveConf, HiveConf.ConfVars.HIVEMAPAGGRHASHMINREDUCTION);
+ GroupByDesc groupBy =
+ new GroupByDesc(gbMode, gbOutputNames, Collections.emptyList(), gbAggs, false, groupByMemoryUsage,
+ memoryThreshold, minReductionHashAggr, null, false, -1, false);
+ groupBy.setColumnExprMap(Collections.emptyMap());
+ return (GroupByOperator) OperatorFactory.getAndMakeChild(groupBy, new RowSchema(gbColInfos), parentOp);
+ }
+
+ private static List<ExprNodeDesc> createGroupByAggregationParameters(SelectOperator selectOp) {
+ List<ExprNodeDesc> params = new ArrayList<>();
+ // The first n-1 cols are used as parameters for min & max so we need two expressions
+ for (ColumnInfo c : selectOp.getSchema().getSignature()) {
+ String name = c.getInternalName();
+ ExprNodeColumnDesc p = new ExprNodeColumnDesc(new ColumnInfo(name, c.getType(), "", false));
+ params.add(p);
+ params.add(p);
+ }
+ // The last col is used as parameter for bloom so we need only one expression
+ params.remove(params.size() - 1);
+ return params;
+ }
+
+ private static List<ExprNodeDesc> createGroupByAggregationParameters(ReduceSinkOperator reduceOp) {
+ List<ExprNodeDesc> params = new ArrayList<>();
+ // There is a 1-1 mapping between columns and parameters for the aggregation functions min, max, bloom
+ for (ColumnInfo c : reduceOp.getSchema().getSignature()) {
+ String name = Utilities.ReduceField.VALUE + "." + c.getInternalName();
+ params.add(new ExprNodeColumnDesc(new ColumnInfo(name, c.getType(), "", false)));
+ }
+ return params;
+ }
+
+ private static AggregationDesc minAggregation(GenericUDAFEvaluator.Mode mode, ExprNodeDesc col) {
+ List<ExprNodeDesc> p = Collections.singletonList(col);
+ return new AggregationDesc("min", new GenericUDAFMin.GenericUDAFMinEvaluator(), p, false, mode);
+ }
+
+ private static AggregationDesc maxAggregation(GenericUDAFEvaluator.Mode mode, ExprNodeDesc col) {
+ List<ExprNodeDesc> p = Collections.singletonList(col);
+ return new AggregationDesc("max", new GenericUDAFMax.GenericUDAFMaxEvaluator(), p, false, mode);
+ }
+
+ private static AggregationDesc bloomFilterAggregation(GenericUDAFEvaluator.Mode mode, ExprNodeDesc col,
+ SelectOperator source, long numEntriesHint, HiveConf conf) {
+ GenericUDAFBloomFilterEvaluator bloomFilterEval = new GenericUDAFBloomFilterEvaluator();
+ bloomFilterEval.setSourceOperator(source);
+ bloomFilterEval.setMaxEntries(conf.getLongVar(HiveConf.ConfVars.TEZ_MAX_BLOOM_FILTER_ENTRIES));
+ bloomFilterEval.setMinEntries(conf.getLongVar(HiveConf.ConfVars.TEZ_MIN_BLOOM_FILTER_ENTRIES));
+ bloomFilterEval.setFactor(conf.getFloatVar(HiveConf.ConfVars.TEZ_BLOOM_FILTER_FACTOR));
+ bloomFilterEval.setHintEntries(numEntriesHint);
+ List<ExprNodeDesc> p = Collections.singletonList(col);
+ AggregationDesc bloom = new AggregationDesc("bloom_filter", bloomFilterEval, p, false, mode);
+ // It is necessary to set the bloom filter evaluator otherwise there are runtime failures see HIVE-24018
+ bloom.setGenericUDAFWritableEvaluator(bloomFilterEval);
+ return bloom;
+ }
+
+ /**
+ * An object that represents the source and target of a semi-join reducer.
+ *
+ * The objects are meant to be used as keys in maps so it is required to implement hashCode and equals methods.
+ */
+ private static final class SJSourceTarget {
+ private final Operator<?> source;
+ private final TableScanOperator target;
+
+ public SJSourceTarget(Operator<?> source, TableScanOperator target) {
+ this.source = source;
+ this.target = target;
+ }
+
+ @Override public boolean equals(Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (o == null || getClass() != o.getClass()) {
+ return false;
+ }
+
+ SJSourceTarget that = (SJSourceTarget) o;
+
+ if (!source.equals(that.source)) {
+ return false;
+ }
+ return target.equals(that.target);
+ }
+
+ @Override public int hashCode() {
+ int result = source.hashCode();
+ result = 31 * result + target.hashCode();
+ return result;
+ }
+
+ }
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java b/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java
index 99b0225..6c36375 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java
@@ -128,7 +128,15 @@ public class ParseContext {
private Map<ReduceSinkOperator, RuntimeValuesInfo> rsToRuntimeValuesInfo =
new LinkedHashMap<ReduceSinkOperator, RuntimeValuesInfo>();
- private Map<ReduceSinkOperator, SemiJoinBranchInfo> rsToSemiJoinBranchInfo = new HashMap<>();
+ /**
+ * Mapping holding information about semijoins.
+ *
+ * In various places we need to iterate over the entries of the map so having a predictable iterator is important for
+ * keeping the plans stable. It is possible to use an alternative implementation that does not provide specific
+ * iteration guarantees but this might introduce small changes in the plans (e.g., different operator ids) from one
+ * execution to the other.
+ */
+ private LinkedHashMap<ReduceSinkOperator, SemiJoinBranchInfo> rsToSemiJoinBranchInfo = new LinkedHashMap<>();
private Map<ExprNodeDesc, GroupByOperator> colExprToGBMap = new HashMap<>();
private Map<String, List<SemiJoinHint>> semiJoinHints;
@@ -670,11 +678,11 @@ public class ParseContext {
return rsToRuntimeValuesInfo;
}
- public void setRsToSemiJoinBranchInfo(Map<ReduceSinkOperator, SemiJoinBranchInfo> rsToSemiJoinBranchInfo) {
+ public void setRsToSemiJoinBranchInfo(LinkedHashMap<ReduceSinkOperator, SemiJoinBranchInfo> rsToSemiJoinBranchInfo) {
this.rsToSemiJoinBranchInfo = rsToSemiJoinBranchInfo;
}
- public Map<ReduceSinkOperator, SemiJoinBranchInfo> getRsToSemiJoinBranchInfo() {
+ public LinkedHashMap<ReduceSinkOperator, SemiJoinBranchInfo> getRsToSemiJoinBranchInfo() {
return rsToSemiJoinBranchInfo;
}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/parse/RuntimeValuesInfo.java b/ql/src/java/org/apache/hadoop/hive/ql/parse/RuntimeValuesInfo.java
index 1fd5e64..531a628 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/parse/RuntimeValuesInfo.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/parse/RuntimeValuesInfo.java
@@ -33,8 +33,10 @@ public class RuntimeValuesInfo implements Serializable {
private TableDesc tableDesc;
private List<String> dynamicValueIDs;
private List<ExprNodeDesc> colExprs;
- // Column expression of the table being filtered by the semijoin optimization.
- private ExprNodeDesc tsColExpr;
+ /**
+ * Column expressions of the (target) table being filtered by the semi-join.
+ */
+ private List<ExprNodeDesc> targetColumns;
// get-set methods
public TableDesc getTableDesc() {
@@ -61,12 +63,12 @@ public class RuntimeValuesInfo implements Serializable {
this.colExprs = colExprs;
}
- public ExprNodeDesc getTsColExpr() {
- return tsColExpr;
+ public List<ExprNodeDesc> getTargetColumns() {
+ return targetColumns;
}
- public void setTsColExpr(ExprNodeDesc tsColExpr) {
- this.tsColExpr = tsColExpr;
+ public void setTargetColumns(List<ExprNodeDesc> targetColumns) {
+ this.targetColumns = targetColumns;
}
}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java b/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
index ce8bf73..ef2b51f 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
@@ -4948,6 +4948,7 @@ public class SemanticAnalyzer extends BaseSemanticAnalyzer {
switch (mode) {
case COMPLETE:
return GenericUDAFEvaluator.Mode.COMPLETE;
+ case HASH:
case PARTIAL1:
return GenericUDAFEvaluator.Mode.PARTIAL1;
case PARTIAL2:
@@ -4957,8 +4958,6 @@ public class SemanticAnalyzer extends BaseSemanticAnalyzer {
: GenericUDAFEvaluator.Mode.PARTIAL2;
case FINAL:
return GenericUDAFEvaluator.Mode.FINAL;
- case HASH:
- return GenericUDAFEvaluator.Mode.PARTIAL1;
case MERGEPARTIAL:
return isDistinct ? GenericUDAFEvaluator.Mode.COMPLETE
: GenericUDAFEvaluator.Mode.FINAL;
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java b/ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java
index f3ff485..7665d5e 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java
@@ -17,8 +17,8 @@
*/
package org.apache.hadoop.hive.ql.parse;
+import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ListMultimap;
-import com.google.common.collect.Sets;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Comparator;
@@ -91,6 +91,7 @@ import org.apache.hadoop.hive.ql.optimizer.MergeJoinProc;
import org.apache.hadoop.hive.ql.optimizer.NonBlockingOpDeDupProc;
import org.apache.hadoop.hive.ql.optimizer.ReduceSinkMapJoinProc;
import org.apache.hadoop.hive.ql.optimizer.RemoveDynamicPruningBySize;
+import org.apache.hadoop.hive.ql.optimizer.SemiJoinReductionMerge;
import org.apache.hadoop.hive.ql.optimizer.SetHashGroupByMinReduction;
import org.apache.hadoop.hive.ql.optimizer.SetReducerParallelism;
import org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer;
@@ -183,6 +184,13 @@ public class TezCompiler extends TaskCompiler {
runDynamicPartitionPruning(procCtx, inputs, outputs);
perfLogger.PerfLogEnd(this.getClass().getName(), PerfLogger.TEZ_COMPILER, "Setup dynamic partition pruning");
+ if(procCtx.conf.getBoolVar(ConfVars.TEZ_DYNAMIC_SEMIJOIN_REDUCTION_MULTICOLUMN)) {
+ SemiJoinReductionMerge sjmerge = new SemiJoinReductionMerge();
+ sjmerge.beginPerfLogging();
+ sjmerge.transform(procCtx.parseContext);
+ sjmerge.endPerfLogging("Merge single column semi-join reducers to composite");
+ }
+
// need to run this; to get consistent filterop conditions(for operator tree matching)
if (procCtx.conf.getBoolVar(ConfVars.HIVEOPTCONSTANTPROPAGATION)) {
new ConstantPropagate(ConstantPropagateOption.SHORTCUT).transform(procCtx.parseContext);
@@ -922,10 +930,7 @@ public class TezCompiler extends TaskCompiler {
Set<ReduceSinkOperator> workRSOps = new HashSet<>();
Set<TerminalOperator<?>> workTerminalOps = new HashSet<>();
// Get the SEL Op in the semijoin-branch, SEL->GBY1->RS1->GBY2->RS2
- Operator<?> selOp = rs.getParentOperators().get(0)
- .getParentOperators().get(0)
- .getParentOperators().get(0)
- .getParentOperators().get(0);
+ SelectOperator selOp = OperatorUtils.ancestor(rs, SelectOperator.class, 0, 0, 0, 0);
OperatorUtils.findWorkOperatorsAndSemiJoinEdges(selOp,
pCtx.getRsToSemiJoinBranchInfo(), workRSOps, workTerminalOps);
@@ -1089,7 +1094,6 @@ public class TezCompiler extends TaskCompiler {
return null;
}
TableScanOperator targetTSOp = sjInfo.getTsOp();
- ExprNodeDesc targetColExpr = pCtx.getRsToRuntimeValuesInfoMap().get(rs).getTsColExpr();
// This is a semijoin branch. The stack should look like,
// <Parent Ops>-SEL-GB1-RS1-GB2-RS2
@@ -1171,9 +1175,11 @@ public class TezCompiler extends TaskCompiler {
// We found siblings, since we are removing the other operator, no need to remove this one
continue;
}
- ExprNodeDesc otherColExpr = pCtx.getRsToRuntimeValuesInfoMap().get(otherRSOp).getTsColExpr();
- if (!otherColExpr.isSame(targetColExpr)) {
- // Filter should be on the same column, otherwise we do not proceed
+ List<ExprNodeDesc> thisTargetColumns = pCtx.getRsToRuntimeValuesInfoMap().get(rs).getTargetColumns();
+ List<ExprNodeDesc> otherTargetColumns =
+ pCtx.getRsToRuntimeValuesInfoMap().get(otherRSOp).getTargetColumns();
+ if (!ExprNodeDescUtils.isSame(thisTargetColumns, otherTargetColumns)) {
+ // Filter should be on the same columns, otherwise we do not proceed
continue;
}
rCtx.opsToRemove.put(rs, targetTSOp);
@@ -1691,21 +1697,15 @@ public class TezCompiler extends TaskCompiler {
return keyDomainCardinality;
}
- private static double getBloomFilterBenefit(
+ private static double getBloomFilterSelectivity(
SelectOperator sel, ExprNodeDesc selExpr,
Statistics filStats, ExprNodeDesc tsExpr) {
- double benefit = -1;
Statistics selStats = sel.getStatistics();
- if (selStats == null || filStats == null) {
- LOG.debug("No stats available to compute BloomFilter benefit");
- return benefit;
- }
-
+ assert selStats != null;
+ assert filStats != null;
// For cardinality values use numRows as default, try to use ColStats if available
long selKeyCardinality = selStats.getNumRows();
long tsKeyCardinality = filStats.getNumRows();
- long tsRows = filStats.getNumRows();
- long tsRowSize = filStats.getAvgRowSize();
long keyDomainCardinality = selKeyCardinality + tsKeyCardinality;
ExprNodeColumnDesc selCol = ExprNodeDescUtils.getColumnExpr(selExpr);
@@ -1717,9 +1717,6 @@ public class TezCompiler extends TaskCompiler {
if (canUseNDV(selColStat)) {
selKeyCardinality = selColStat.getCountDistint();
}
- if (canUseNDV(filColStat)) {
- tsKeyCardinality = filColStat.getCountDistint();
- }
// Get colstats for the original table column for selCol if possible, this would have
// more accurate information about the original NDV of the column before any filtering.
ColStatistics selColSourceStat = null;
@@ -1737,35 +1734,54 @@ public class TezCompiler extends TaskCompiler {
}
}
- // Selectivity: key cardinality of semijoin / domain cardinality
- // Benefit (rows filtered from ts): (1 - selectivity) * # ts rows
- double selectivity = selKeyCardinality / (double) keyDomainCardinality;
- selectivity = Math.min(selectivity, 1);
- benefit = tsRows * (1 - selectivity);
-
if (LOG.isDebugEnabled()) {
- LOG.debug("BloomFilter benefit for " + selCol + " to " + tsCol
- + ", selKeyCardinality=" + selKeyCardinality
- + ", tsKeyCardinality=" + tsKeyCardinality
- + ", tsRows=" + tsRows
- + ", keyDomainCardinality=" + keyDomainCardinality);
- LOG.debug("SemiJoin key selectivity=" + selectivity
- + ", benefit=" + benefit);
+ LOG.debug("BloomFilter selectivity for " + selCol + " to " + tsCol + ", selKeyCardinality=" + selKeyCardinality
+ + ", tsKeyCardinality=" + tsKeyCardinality + ", keyDomainCardinality=" + keyDomainCardinality);
}
+ // Selectivity: key cardinality of semijoin / domain cardinality
+ return selKeyCardinality / (double) keyDomainCardinality;
+ }
- return benefit;
+ /**
+ * Computes the benefit of applying the bloom filter.
+ * <p>
+ * The benefit is defined as the number of tuples that are filtered/removed from the bloom.
+ * </p>
+ */
+ private static double getBloomFilterBenefit(SelectOperator sel, List<ExprNodeDesc> selExpr, Statistics filStats,
+ List<ExprNodeDesc> tsExpr) {
+ if (sel.getStatistics() == null || filStats == null) {
+ LOG.debug("No stats available to compute BloomFilter benefit");
+ return -1;
+ }
+ // Find the semijoin column with the smallest number of matches and keep its selectivity
+ double selectivity = 1.0;
+ for (int i = 0; i < tsExpr.size(); i++) {
+ selectivity = Math.min(selectivity, getBloomFilterSelectivity(sel, selExpr.get(i), filStats, tsExpr.get(i)));
+ }
+ // Decrease the min selectivity by 5% for each additional column in the semijoin.
+ // Consider the following semijoins:
+ // SJ1(author.name, author.age);
+ // SJ2(author.name).
+ // Intuitively even if the min selectivity of both is 0.8 the semijoin with two columns (SJ1)
+ // will match less tuples than the semijoin with one column (SJ2).
+ selectivity -= selectivity * (tsExpr.size() - 1) * 0.05;
+ // Selectivity cannot be less than 0.0
+ selectivity = Math.max(0.0, selectivity);
+ // Benefit (rows filtered from ts): (1 - selectivity) * # ts rows
+ return filStats.getNumRows() * (1 - selectivity);
}
private static double computeBloomFilterNetBenefit(
- SelectOperator sel, ExprNodeDesc selExpr,
- Statistics filStats, ExprNodeDesc tsExpr) {
- double netBenefit = -1;
+ SelectOperator sel, List<ExprNodeDesc> selExpr,
+ Statistics filStats, List<ExprNodeDesc> tsExpr) {
+ double netBenefit = 0;
double benefit = getBloomFilterBenefit(sel, selExpr, filStats, tsExpr);
if (benefit > 0 && filStats != null) {
double cost = getBloomFilterCost(sel);
if (cost > 0) {
long filDataSize = filStats.getNumRows();
- netBenefit = (benefit - cost) / filDataSize;
+ netBenefit = Math.max(benefit - cost, 0) / filDataSize;
LOG.debug("BloomFilter benefit=" + benefit
+ ", cost=" + cost
+ ", tsDataSize=" + filDataSize
@@ -1872,28 +1888,22 @@ public class TezCompiler extends TaskCompiler {
continue;
}
// rs is semijoin optimization branch, which should look like <Parent>-SEL-GB1-RS1-GB2-RS2
- // Get to the SelectOperator ancestor
- SelectOperator sel = null;
- for (Operator<?> currOp = rs; currOp.getParentOperators().size() > 0; currOp = currOp.getParentOperators().get(0)) {
- if (currOp instanceof SelectOperator) {
- sel = (SelectOperator) currOp;
- break;
- }
- }
- if (sel == null) {
- throw new SemanticException("Unexpected error - could not find SEL ancestor from semijoin branch of " + rs);
- }
+ SelectOperator sel = OperatorUtils.ancestor(rs, SelectOperator.class, 0, 0, 0, 0);
// Check the ndv/rows from the SEL vs the destination tablescan the semijoin opt is going to.
TableScanOperator ts = sjInfo.getTsOp();
RuntimeValuesInfo rti = procCtx.parseContext.getRsToRuntimeValuesInfoMap().get(rs);
- ExprNodeDesc tsExpr = rti.getTsColExpr();
- // In the SEL operator of the semijoin branch, there should be only one column in the operator
- ExprNodeDesc selExpr = sel.getConf().getColList().get(0);
+ List<ExprNodeDesc> targetColumns = rti.getTargetColumns();
+ // In semijoin branches the SEL operator has the following forms:
+ // SEL[c1] - single column semijoin reduction
+ // SEL[c1, c2,..., ck, hash(c1, c2,...,ck)] - multi column semijoin reduction
+ // The source columns in the above cases are c1, c2,...,ck.
+ // We need to exclude the hash(...) expression, if it is present.
+ List<ExprNodeDesc> sourceColumns = sel.getConf().getColList().subList(0, targetColumns.size());
if (LOG.isDebugEnabled()) {
LOG.debug("Computing BloomFilter cost/benefit for " + OperatorUtils.getOpNamePretty(rs)
- + " - " + OperatorUtils.getOpNamePretty(ts) + " (" + tsExpr + ")");
+ + " - " + OperatorUtils.getOpNamePretty(ts) + " " + targetColumns + " ");
}
FilterOperator filterOperator = (FilterOperator) ts.getChildOperators().get(0);
@@ -1902,21 +1912,22 @@ public class TezCompiler extends TaskCompiler {
filterStats = filterOperator.getStatistics().clone();
adjustedStatsMap.put(filterOperator, filterStats);
}
- double reductionFactor = computeBloomFilterNetBenefit(
- sel, selExpr, filterStats, tsExpr);
+ double reductionFactor = computeBloomFilterNetBenefit(sel, sourceColumns, filterStats, targetColumns);
if (reductionFactor < semijoinReductionThreshold) {
// This semijoin optimization should be removed. Do it after we're done iterating
semijoinRsToRemove.add(rs);
} else {
// This semijoin qualifies, add it to the result set
if (filterStats != null) {
- // tsExpr might actually be a ExprNodeFieldDesc and we need to extract the column expression
- if (tsExpr instanceof ExprNodeFieldDesc) {
- LOG.info("Unwrapped column expression from ExprNodeFieldDesc");
- tsExpr = ((ExprNodeFieldDesc)tsExpr).getDesc();
+ ImmutableSet.Builder<String> colNames = ImmutableSet.builder();
+ for (ExprNodeDesc tsExpr : targetColumns) {
+ // tsExpr might actually be a ExprNodeFieldDesc and we need to extract the column expression
+ if (tsExpr instanceof ExprNodeFieldDesc) {
+ LOG.debug("Unwrapped column expression from ExprNodeFieldDesc");
+ tsExpr = ((ExprNodeFieldDesc) tsExpr).getDesc();
+ }
+ colNames.add(ExprNodeDescUtils.getColumnExpr(tsExpr).getColumn());
}
-
- String colName = ExprNodeDescUtils.getColumnExpr(tsExpr).getColumn();
// We check whether there was already another SJ over this TS that was selected
// in previous iteration
SemijoinOperatorInfo prevResult = reductionFactorMap.get(filterOperator);
@@ -1926,7 +1937,7 @@ public class TezCompiler extends TaskCompiler {
// that we found. We add the previous RS where SJ was originating to RS ops for new
// iteration
reductionFactorMap.put(filterOperator, new SemijoinOperatorInfo(rs, filterOperator,
- filterStats, colName, reductionFactor));
+ filterStats, colNames.build(), reductionFactor));
semiJoinRsOpsNewIter.add(prevResult.rsOperator);
if (LOG.isDebugEnabled()) {
LOG.debug("Adding " + OperatorUtils.getOpNamePretty(prevResult.rsOperator)
@@ -1943,7 +1954,7 @@ public class TezCompiler extends TaskCompiler {
} else {
// Another SJ did not exist for this TS, hence just add it to SJs to keep
reductionFactorMap.put(filterOperator, new SemijoinOperatorInfo(rs, filterOperator,
- filterStats, colName, reductionFactor));
+ filterStats, colNames.build(), reductionFactor));
}
}
}
@@ -1958,7 +1969,7 @@ public class TezCompiler extends TaskCompiler {
LOG.debug("Number of rows reduction: {}/{}", newNumRows, roi.filterStats.getNumRows());
}
StatsUtils.updateStats(roi.filterStats, newNumRows,
- true, roi.filterOperator, Sets.newHashSet(roi.colName));
+ true, roi.filterOperator, roi.colNames);
if (LOG.isDebugEnabled()) {
LOG.debug("New stats for {}: {}", roi.filterOperator, roi.filterStats);
}
@@ -1991,15 +2002,15 @@ public class TezCompiler extends TaskCompiler {
private class SemijoinOperatorInfo {
final ReduceSinkOperator rsOperator;
final FilterOperator filterOperator;
- final String colName;
+ final ImmutableSet<String> colNames;
final Statistics filterStats;
final double reductionFactor;
private SemijoinOperatorInfo(ReduceSinkOperator rsOperator, FilterOperator filterOperator,
- Statistics filterStats, String colName, double reductionFactor) {
+ Statistics filterStats, Collection<String> colNames, double reductionFactor) {
this.rsOperator = rsOperator;
this.filterOperator = filterOperator;
- this.colName = colName;
+ this.colNames = ImmutableSet.copyOf(colNames);
this.filterStats = filterStats;
this.reductionFactor = reductionFactor;
}
@@ -2028,11 +2039,7 @@ public class TezCompiler extends TaskCompiler {
((AppMasterEventOperator) op).getConf() instanceof DynamicPruningEventDesc) {
// DPP. Now look up nDVs on both sides to see the selectivity.
// <Parent Ops>-SEL-GB1-RS1-GB2-RS2
- SelectOperator selOp = (SelectOperator)
- (rs.getParentOperators().get(0)
- .getParentOperators().get(0)
- .getParentOperators().get(0)
- .getParentOperators().get(0));
+ SelectOperator selOp = OperatorUtils.ancestor(rs, SelectOperator.class, 0, 0, 0, 0);
try {
// Get nDVs on Semijoin edge side
@@ -2054,7 +2061,8 @@ public class TezCompiler extends TaskCompiler {
// Lookup nDVs on TS side.
RuntimeValuesInfo rti = procCtx.parseContext
.getRsToRuntimeValuesInfoMap().get(rs);
- ExprNodeDesc tsExpr = rti.getTsColExpr();
+ // TODO Handle multi column semi-joins as part of HIVE-23934
+ ExprNodeDesc tsExpr = rti.getTargetColumns().get(0);
FilterOperator fil = (FilterOperator) (ts.getChildOperators().get(0));
Statistics filStats = fil.getStatistics();
if (filStats == null) {
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java b/ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java
index bd92730..0093759 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java
@@ -19,8 +19,10 @@
package org.apache.hadoop.hive.ql.plan;
import com.google.common.collect.Multimap;
+
+import java.util.Arrays;
import java.util.Collection;
-import org.apache.hadoop.hive.ql.parse.type.TypeCheckProcFactory;
+import org.apache.hadoop.hive.ql.udf.generic.GenericUDFOPAnd;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.hadoop.hive.ql.exec.ColumnInfo;
@@ -204,6 +206,33 @@ public class ExprNodeDescUtils {
return false;
}
+
+ /**
+ * Creates a conjunction (AND) of two expressions flattening nested conjunctions if possible.
+ * <p>
+ * The method is equivalent to calling: {@code and(Arrays.asList(e1, e2))}
+ * </p>
+ */
+ public static ExprNodeGenericFuncDesc and(ExprNodeDesc e1, ExprNodeDesc e2) {
+ return and(Arrays.asList(e1, e2));
+ }
+
+ /**
+ * Creates a conjunction (AND) of the given expressions flattening nested conjunctions if possible.
+ * <pre>
+ * Input: OR(A, B), C, AND(D, AND(E, F))
+ * Output: AND(OR(A, B), C, D, E, F)
+ * </pre>
+ * TODO: Replace mergePredicates ?
+ */
+ public static ExprNodeGenericFuncDesc and(List<ExprNodeDesc> exps) {
+ List<ExprNodeDesc> flatExps = new ArrayList<>();
+ for (ExprNodeDesc e : exps) {
+ split(e, flatExps);
+ }
+ return new ExprNodeGenericFuncDesc(TypeInfoFactory.booleanTypeInfo, new GenericUDFOPAnd(), "and", flatExps);
+ }
+
/**
* bind two predicates by AND op
*/
diff --git a/ql/src/test/queries/clientpositive/dynamic_semijoin_reduction_multicol.q b/ql/src/test/queries/clientpositive/dynamic_semijoin_reduction_multicol.q
new file mode 100644
index 0000000..df5c64d
--- /dev/null
+++ b/ql/src/test/queries/clientpositive/dynamic_semijoin_reduction_multicol.q
@@ -0,0 +1,94 @@
+--! qt:dataset:tpch_0_001.lineitem
+--! qt:dataset:tpch_0_001.partsupp
+use tpch_0_001;
+-- The test is meant to verify the plan, results, and Tez execution stats/counters
+-- of the same query in the following cases:
+-- case 1: no semi-join reducers
+-- case 2: one single column semi-join reducer
+-- case 3: one multi column semi-join reducer
+-- case 4: one multi column semi-join reducer created via hints
+set hive.mapred.mode=nonstrict;
+set hive.explain.user=false;
+set hive.tez.bigtable.minsize.semijoin.reduction=6000;
+-- Use TezSummaryPrinter hook to verify the impact of bloom filters by
+-- comparing the RECORDS_OUT of the respective operators.
+set hive.exec.post.hooks=org.apache.hadoop.hive.ql.hooks.PostExecTezSummaryPrinter;
+
+-- Case 1: No semi-join reducers in the query
+set hive.tez.dynamic.semijoin.reduction=false;
+
+explain
+select li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+from lineitem li
+ inner join partsupp ps on li.l_partkey = ps.ps_partkey and li.l_suppkey = ps.ps_suppkey
+where ps.ps_availqty < 15
+order by li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty;
+
+select li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+from lineitem li
+ inner join partsupp ps on li.l_partkey = ps.ps_partkey and li.l_suppkey = ps.ps_suppkey
+where ps.ps_availqty < 15
+order by li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty;
+
+-- Case 2: One single column semi-join reducer
+set hive.tez.dynamic.semijoin.reduction=true;
+set hive.tez.dynamic.semijoin.reduction.multicolumn=false;
+
+explain
+select li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+from lineitem li
+ inner join partsupp ps on li.l_partkey = ps.ps_partkey and li.l_suppkey = ps.ps_suppkey
+where ps.ps_availqty < 15
+order by li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty;
+
+select li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+from lineitem li
+ inner join partsupp ps on li.l_partkey = ps.ps_partkey and li.l_suppkey = ps.ps_suppkey
+where ps.ps_availqty < 15
+order by li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty;
+
+-- Case 3: One multi-column semi-join reducer
+set hive.tez.dynamic.semijoin.reduction=true;
+set hive.tez.dynamic.semijoin.reduction.multicolumn=true;
+
+explain
+select li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+from lineitem li
+ inner join partsupp ps on li.l_partkey = ps.ps_partkey and li.l_suppkey = ps.ps_suppkey
+where ps.ps_availqty < 15
+order by li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty;
+
+select li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+from lineitem li
+ inner join partsupp ps on li.l_partkey = ps.ps_partkey and li.l_suppkey = ps.ps_suppkey
+where ps.ps_availqty < 15
+order by li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty;
+
+-- Set the size of the big table to a value much greater than the real sizes of
+-- the tables so that semijoin reducers do not kick in without explicit hints
+set hive.tez.bigtable.minsize.semijoin.reduction=100000000;
+-- Set the expected size of the bloom filter in the columns to different values
+-- so that we can verify that the plan picks the bigger one for the composite
+-- bloom filter.
+explain
+select
+ /*+ semi(ps, ps_partkey, li, 2), semi(ps, ps_suppkey, li, 3)*/
+ li.l_orderkey,
+ li.l_partkey,
+ li.l_suppkey,
+ ps.ps_availqty
+from lineitem li
+ inner join partsupp ps on li.l_partkey = ps.ps_partkey and li.l_suppkey = ps.ps_suppkey
+where ps.ps_availqty < 15
+order by li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty;
+
+select
+ /*+ semi(ps, ps_partkey, li, 2), semi(ps, ps_suppkey, li, 3)*/
+ li.l_orderkey,
+ li.l_partkey,
+ li.l_suppkey,
+ ps.ps_availqty
+from lineitem li
+ inner join partsupp ps on li.l_partkey = ps.ps_partkey and li.l_suppkey = ps.ps_suppkey
+where ps.ps_availqty < 15
+order by li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty;
\ No newline at end of file
diff --git a/ql/src/test/queries/clientpositive/vectorized_dynamic_semijoin_reduction.q b/ql/src/test/queries/clientpositive/vectorized_dynamic_semijoin_reduction.q
index 00c9332..953ecbd 100644
--- a/ql/src/test/queries/clientpositive/vectorized_dynamic_semijoin_reduction.q
+++ b/ql/src/test/queries/clientpositive/vectorized_dynamic_semijoin_reduction.q
@@ -35,6 +35,8 @@ EXPLAIN VECTORIZATION EXPRESSION select count(*) from dsrv_big a, dsrv_small b,
select count(*) from dsrv_big a, dsrv_small b, dsrv_small c where a.key_int = b.key_int and a.key_int = c.key_int;
-- multiple keys
+-- When hive.tez.dynamic.semijoin.reduction.multicolumn is active and applied to the plan vectorized execution cannot be
+-- performed. We plan to address this case in HIVE-23976.
EXPLAIN VECTORIZATION EXPRESSION select count(*) from dsrv_big a join dsrv_small b on (a.key_str = b.key_str and a.key_int = b.key_int);
select count(*) from dsrv_big a join dsrv_small b on (a.key_str = b.key_str and a.key_int = b.key_int);
diff --git a/ql/src/test/results/clientpositive/llap/acid_stats3.q.out b/ql/src/test/results/clientpositive/llap/acid_stats3.q.out
index 83d52c9..6ba2a8f 100644
--- a/ql/src/test/results/clientpositive/llap/acid_stats3.q.out
+++ b/ql/src/test/results/clientpositive/llap/acid_stats3.q.out
@@ -81,7 +81,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: may be used (ACID table)
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -267,7 +267,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 12 Basic stats: PARTIAL Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: may be used (ACID table)
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -385,7 +385,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 12 Basic stats: PARTIAL Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: may be used (ACID table)
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
diff --git a/ql/src/test/results/clientpositive/llap/acid_stats4.q.out b/ql/src/test/results/clientpositive/llap/acid_stats4.q.out
index 2b6cba4..fb085c3 100644
--- a/ql/src/test/results/clientpositive/llap/acid_stats4.q.out
+++ b/ql/src/test/results/clientpositive/llap/acid_stats4.q.out
@@ -83,7 +83,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: may be used (ACID table)
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -682,7 +682,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Execution mode: llap
- LLAP IO: no inputs
+ LLAP IO: may be used (ACID table)
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -819,7 +819,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 12 Basic stats: PARTIAL Column stats: PARTIAL
value expressions: _col0 (type: bigint)
Execution mode: llap
- LLAP IO: no inputs
+ LLAP IO: may be used (ACID table)
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -917,7 +917,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 12 Basic stats: PARTIAL Column stats: PARTIAL
value expressions: _col0 (type: bigint)
Execution mode: llap
- LLAP IO: no inputs
+ LLAP IO: may be used (ACID table)
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -995,7 +995,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 12 Basic stats: PARTIAL Column stats: PARTIAL
value expressions: _col0 (type: bigint)
Execution mode: llap
- LLAP IO: no inputs
+ LLAP IO: may be used (ACID table)
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
diff --git a/ql/src/test/results/clientpositive/llap/acid_stats5.q.out b/ql/src/test/results/clientpositive/llap/acid_stats5.q.out
index 8a891eb..774883e 100644
--- a/ql/src/test/results/clientpositive/llap/acid_stats5.q.out
+++ b/ql/src/test/results/clientpositive/llap/acid_stats5.q.out
@@ -165,7 +165,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: may be used (ACID table)
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -231,7 +231,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: may be used (ACID table)
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -396,7 +396,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int)
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: may be used (ACID table)
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -644,7 +644,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: may be used (ACID table)
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -809,7 +809,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: int)
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: may be used (ACID table)
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
diff --git a/ql/src/test/results/clientpositive/llap/allcolref_in_udf.q.out b/ql/src/test/results/clientpositive/llap/allcolref_in_udf.q.out
index a50f9ad..9a9b857 100644
--- a/ql/src/test/results/clientpositive/llap/allcolref_in_udf.q.out
+++ b/ql/src/test/results/clientpositive/llap/allcolref_in_udf.q.out
@@ -93,7 +93,7 @@ STAGE PLANS:
Statistics: Num rows: 166 Data size: 61420 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: all inputs
Map 3
Map Operator Tree:
TableScan
@@ -115,7 +115,7 @@ STAGE PLANS:
Statistics: Num rows: 500 Data size: 185000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
diff --git a/ql/src/test/results/clientpositive/llap/alterColumnStatsPart.q.out b/ql/src/test/results/clientpositive/llap/alterColumnStatsPart.q.out
index 2e2b391..678c542 100644
--- a/ql/src/test/results/clientpositive/llap/alterColumnStatsPart.q.out
+++ b/ql/src/test/results/clientpositive/llap/alterColumnStatsPart.q.out
@@ -159,7 +159,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: int)
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -317,7 +317,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: int)
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -428,7 +428,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: PARTIAL Column stats: NONE
value expressions: _col0 (type: int)
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -509,7 +509,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: PARTIAL Column stats: NONE
value expressions: _col0 (type: int)
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
diff --git a/ql/src/test/results/clientpositive/llap/ambiguous_col.q.out b/ql/src/test/results/clientpositive/llap/ambiguous_col.q.out
index 1ff7d79..2d16085 100644
--- a/ql/src/test/results/clientpositive/llap/ambiguous_col.q.out
+++ b/ql/src/test/results/clientpositive/llap/ambiguous_col.q.out
@@ -41,7 +41,7 @@ STAGE PLANS:
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: all inputs
Map 3
Map Operator Tree:
TableScan
@@ -62,7 +62,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -181,7 +181,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: all inputs
Map 3
Map Operator Tree:
TableScan
@@ -202,7 +202,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -321,7 +321,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: all inputs
Map 3
Map Operator Tree:
TableScan
@@ -342,7 +342,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
diff --git a/ql/src/test/results/clientpositive/llap/analyze_tbl_date.q.out b/ql/src/test/results/clientpositive/llap/analyze_tbl_date.q.out
index 5a527ca..8ae9730 100644
--- a/ql/src/test/results/clientpositive/llap/analyze_tbl_date.q.out
+++ b/ql/src/test/results/clientpositive/llap/analyze_tbl_date.q.out
@@ -97,7 +97,7 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: all inputs
Stage: Stage-0
Fetch Operator
diff --git a/ql/src/test/results/clientpositive/llap/annotate_stats_deep_filters.q.out b/ql/src/test/results/clientpositive/llap/annotate_stats_deep_filters.q.out
index 8e5ccfe..02d3cd2 100644
--- a/ql/src/test/results/clientpositive/llap/annotate_stats_deep_filters.q.out
+++ b/ql/src/test/results/clientpositive/llap/annotate_stats_deep_filters.q.out
@@ -149,7 +149,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -255,7 +255,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Execution mode: vectorized, llap
- LLAP IO: no inputs
+ LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
diff --git a/ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out b/ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out
index 46c52b5..7ce121a 100644
--- a/ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out
+++ b/ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out
@@ -1160,10 +1160,10 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart_date_n7
- filterExpr: (key is not null and value is not null and key BETWEEN DynamicValue(RS_7_srcpart_small_n3_key1_min) AND DynamicValue(RS_7_srcpart_small_n3_key1_max) and in_bloom_filter(key, DynamicValue(RS_7_srcpart_small_n3_key1_bloom_filter))) (type: boolean)
+ filterExpr: (key is not null and value is not null and key BETWEEN DynamicValue(RS[43]_col0) AND DynamicValue(RS[43]_col1) and value BETWEEN DynamicValue(RS[43]_col2) AND DynamicValue(RS[43]_col3) and in_bloom_filter(hash(key,value), DynamicValue(RS[43]_col4))) (type: boolean)
Statistics: Num rows: 2000 Data size: 356000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: (key is not null and value is not null and key BETWEEN DynamicValue(RS_7_srcpart_small_n3_key1_min) AND DynamicValue(RS_7_srcpart_small_n3_key1_max) and in_bloom_filter(key, DynamicValue(RS_7_srcpart_small_n3_key1_bloom_filter))) (type: boolean)
+ predicate: (key is not null and value is not null and key BETWEEN DynamicValue(RS[43]_col0) AND DynamicValue(RS[43]_col1) and value BETWEEN DynamicValue(RS[43]_col2) AND DynamicValue(RS[43]_col3) and in_bloom_filter(hash(key,value), DynamicValue(RS[43]_col4))) (type: boolean)
Statistics: Num rows: 2000 Data size: 356000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
@@ -1197,20 +1197,20 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
Statistics: Num rows: 20 Data size: 3560 Basic stats: PARTIAL Column stats: PARTIAL
Select Operator
- expressions: _col0 (type: string)
- outputColumnNames: _col0
+ expressions: _col0 (type: string), _col1 (type: string), hash(_col0,_col1) (type: int)
+ outputColumnNames: _col0, _col1, _col3
Statistics: Num rows: 20 Data size: 3560 Basic stats: PARTIAL Column stats: PARTIAL
Group By Operator
- aggregations: min(_col0), max(_col0), bloom_filter(_col0, expectedEntries=20)
+ aggregations: min(_col0), max(_col0), min(_col1), max(_col1), bloom_filter(_col3, expectedEntries=20)
minReductionHashAggr: 0.99
mode: hash
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 730 Basic stats: PARTIAL Column stats: PARTIAL
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4
+ Statistics: Num rows: 1 Data size: 1058 Basic stats: PARTIAL Column stats: PARTIAL
Reduce Output Operator
null sort order:
sort order:
- Statistics: Num rows: 1 Data size: 730 Basic stats: PARTIAL Column stats: PARTIAL
- value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: binary)
+ Statistics: Num rows: 1 Data size: 1058 Basic stats: PARTIAL Column stats: PARTIAL
+ value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: binary)
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
@@ -1253,15 +1253,15 @@ STAGE PLANS:
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
- aggregations: min(VALUE._col0), max(VALUE._col1), bloom_filter(VALUE._col2, expectedEntries=20)
+ aggregations: min(VALUE._col0), max(VALUE._col1), min(VALUE._col2), max(VALUE._col3), bloom_filter(VALUE._col4, expectedEntries=20)
mode: final
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 730 Basic stats: PARTIAL Column stats: PARTIAL
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4
+ Statistics: Num rows: 1 Data size: 1058 Basic stats: PARTIAL Column stats: PARTIAL
Reduce Output Operator
null sort order:
sort order:
- Statistics: Num rows: 1 Data size: 730 Basic stats: PARTIAL Column stats: PARTIAL
- value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: binary)
+ Statistics: Num rows: 1 Data size: 1058 Basic stats: PARTIAL Column stats: PARTIAL
+ value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: binary)
Stage: Stage-0
Fetch Operator
diff --git a/ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_2.q.out b/ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_2.q.out
index ab7dd5a..c3c1baf 100644
--- a/ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_2.q.out
+++ b/ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_2.q.out
@@ -79,27 +79,25 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Map 1 <- Reducer 11 (BROADCAST_EDGE), Reducer 6 (BROADCAST_EDGE), Reducer 7 (BROADCAST_EDGE), Reducer 8 (BROADCAST_EDGE)
- Map 9 <- Reducer 13 (BROADCAST_EDGE)
- Reducer 10 <- Map 12 (SIMPLE_EDGE), Map 9 (SIMPLE_EDGE)
- Reducer 11 <- Reducer 10 (CUSTOM_SIMPLE_EDGE)
- Reducer 13 <- Map 12 (CUSTOM_SIMPLE_EDGE)
+ Map 1 <- Reducer 6 (BROADCAST_EDGE), Reducer 9 (BROADCAST_EDGE)
+ Map 7 <- Reducer 11 (BROADCAST_EDGE)
+ Reducer 11 <- Map 10 (CUSTOM_SIMPLE_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
- Reducer 3 <- Reducer 10 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
Reducer 4 <- Reducer 3 (CUSTOM_SIMPLE_EDGE)
Reducer 6 <- Map 5 (CUSTOM_SIMPLE_EDGE)
- Reducer 7 <- Map 5 (CUSTOM_SIMPLE_EDGE)
- Reducer 8 <- Map 5 (CUSTOM_SIMPLE_EDGE)
+ Reducer 8 <- Map 10 (SIMPLE_EDGE), Map 7 (SIMPLE_EDGE)
+ Reducer 9 <- Reducer 8 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: t1
- filterExpr: (bigint_col_7 is not null and decimal2016_col_26 is not null and tinyint_col_3 is not null and timestamp_col_9 is not null and decimal2016_col_26 BETWEEN DynamicValue(RS_17_t2_decimal2709_col_9_min) AND DynamicValue(RS_17_t2_decimal2709_col_9_max) and tinyint_col_3 BETWEEN DynamicValue(RS_17_t2_tinyint_col_20_min) AND DynamicValue(RS_17_t2_tinyint_col_20_max) and bigint_col_7 BETWEEN DynamicValue(RS_17_t2_tinyint_col_15_min) AND DynamicValue(RS_17_t2_tinyint [...]
+ filterExpr: (bigint_col_7 is not null and decimal2016_col_26 is not null and tinyint_col_3 is not null and timestamp_col_9 is not null and timestamp_col_9 BETWEEN DynamicValue(RS_22_tt2_timestamp_col_18_min) AND DynamicValue(RS_22_tt2_timestamp_col_18_max) and in_bloom_filter(timestamp_col_9, DynamicValue(RS_22_tt2_timestamp_col_18_bloom_filter)) and decimal2016_col_26 BETWEEN DynamicValue(RS[102]_col0) AND DynamicValue(RS[102]_col1) and tinyint_col_3 BETWEEN DynamicVal [...]
Statistics: Num rows: 1 Data size: 164 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (bigint_col_7 is not null and decimal2016_col_26 is not null and tinyint_col_3 is not null and timestamp_col_9 is not null and decimal2016_col_26 BETWEEN DynamicValue(RS_17_t2_decimal2709_col_9_min) AND DynamicValue(RS_17_t2_decimal2709_col_9_max) and tinyint_col_3 BETWEEN DynamicValue(RS_17_t2_tinyint_col_20_min) AND DynamicValue(RS_17_t2_tinyint_col_20_max) and bigint_col_7 BETWEEN DynamicValue(RS_17_t2_tinyint_col_15_min) AND DynamicValue(RS_17_t2_tinyin [...]
+ predicate: (bigint_col_7 is not null and decimal2016_col_26 is not null and tinyint_col_3 is not null and timestamp_col_9 is not null and decimal2016_col_26 BETWEEN DynamicValue(RS[102]_col0) AND DynamicValue(RS[102]_col1) and tinyint_col_3 BETWEEN DynamicValue(RS[102]_col2) AND DynamicValue(RS[102]_col3) and bigint_col_7 BETWEEN DynamicValue(RS[102]_col4) AND DynamicValue(RS[102]_col5) and timestamp_col_9 BETWEEN DynamicValue(RS_22_tt2_timestamp_col_18_min) AND Dynam [...]
Statistics: Num rows: 1 Data size: 164 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: bigint_col_7 (type: bigint), decimal2016_col_26 (type: decimal(20,16)), tinyint_col_3 (type: tinyint), timestamp_col_9 (type: timestamp)
@@ -114,7 +112,7 @@ STAGE PLANS:
value expressions: _col3 (type: timestamp)
Execution mode: vectorized, llap
LLAP IO: all inputs
- Map 12
+ Map 10
Map Operator Tree:
TableScan
alias: tt1
@@ -171,53 +169,23 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 124 Basic stats: COMPLETE Column stats: NONE
value expressions: _col3 (type: smallint)
Select Operator
- expressions: _col1 (type: decimal(34,16))
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 124 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: min(_col0), max(_col0), bloom_filter(_col0, expectedEntries=1)
- minReductionHashAggr: 0.99
- mode: hash
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 460 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- null sort order:
- sort order:
- Statistics: Num rows: 1 Data size: 460 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: decimal(34,16)), _col1 (type: decimal(34,16)), _col2 (type: binary)
- Select Operator
- expressions: _col2 (type: tinyint)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 124 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: min(_col0), max(_col0), bloom_filter(_col0, expectedEntries=1)
- minReductionHashAggr: 0.99
- mode: hash
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 136 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- null sort order:
- sort order:
- Statistics: Num rows: 1 Data size: 136 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: tinyint), _col1 (type: tinyint), _col2 (type: binary)
- Select Operator
- expressions: UDFToLong(_col0) (type: bigint)
- outputColumnNames: _col0
+ expressions: _col1 (type: decimal(34,16)), _col2 (type: tinyint), UDFToLong(_col0) (type: bigint), hash(_col1,_col2,UDFToLong(_col0)) (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col4
Statistics: Num rows: 1 Data size: 124 Basic stats: COMPLETE Column stats: NONE
Group By Operator
- aggregations: min(_col0), max(_col0), bloom_filter(_col0, expectedEntries=1)
+ aggregations: min(_col0), max(_col0), min(_col1), max(_col1), min(_col2), max(_col2), bloom_filter(_col4, expectedEntries=1)
minReductionHashAggr: 0.99
mode: hash
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 148 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
+ Statistics: Num rows: 1 Data size: 516 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
null sort order:
sort order:
- Statistics: Num rows: 1 Data size: 148 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint), _col1 (type: bigint), _col2 (type: binary)
+ Statistics: Num rows: 1 Data size: 516 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: decimal(34,16)), _col1 (type: decimal(34,16)), _col2 (type: tinyint), _col3 (type: tinyint), _col4 (type: bigint), _col5 (type: bigint), _col6 (type: binary)
Execution mode: vectorized, llap
LLAP IO: all inputs
- Map 9
+ Map 7
Map Operator Tree:
TableScan
alias: tt2
@@ -239,48 +207,6 @@ STAGE PLANS:
value expressions: _col1 (type: timestamp)
Execution mode: vectorized, llap
LLAP IO: all inputs
- Reducer 10
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: decimal(26,12))
- 1 _col0 (type: decimal(26,12))
- outputColumnNames: _col1
- Statistics: Num rows: 1 Data size: 167 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col1 (type: timestamp), -92 (type: int)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 167 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- keys: _col0 (type: timestamp), _col1 (type: int)
- minReductionHashAggr: 0.99
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 167 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: timestamp), _col1 (type: int)
- null sort order: zz
- sort order: ++
- Map-reduce partition columns: _col0 (type: timestamp), _col1 (type: int)
- Statistics: Num rows: 1 Data size: 167 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: timestamp)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 167 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: min(_col0), max(_col0), bloom_filter(_col0, expectedEntries=1)
- minReductionHashAggr: 0.99
- mode: hash
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 120 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- null sort order:
- sort order:
- Statistics: Num rows: 1 Data size: 120 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: timestamp), _col1 (type: timestamp), _col2 (type: binary)
Reducer 11
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -288,19 +214,6 @@ STAGE PLANS:
aggregations: min(VALUE._col0), max(VALUE._col1), bloom_filter(VALUE._col2, expectedEntries=1)
mode: final
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 120 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- null sort order:
- sort order:
- Statistics: Num rows: 1 Data size: 120 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: timestamp), _col1 (type: timestamp), _col2 (type: binary)
- Reducer 13
- Execution mode: vectorized, llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: min(VALUE._col0), max(VALUE._col1), bloom_filter(VALUE._col2, expectedEntries=1)
- mode: final
- outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 1 Data size: 448 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
null sort order:
@@ -364,41 +277,70 @@ STAGE PLANS:
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
- aggregations: min(VALUE._col0), max(VALUE._col1), bloom_filter(VALUE._col2, expectedEntries=1)
+ aggregations: min(VALUE._col0), max(VALUE._col1), min(VALUE._col2), max(VALUE._col3), min(VALUE._col4), max(VALUE._col5), bloom_filter(VALUE._col6, expectedEntries=1)
mode: final
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 460 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
+ Statistics: Num rows: 1 Data size: 516 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
null sort order:
sort order:
- Statistics: Num rows: 1 Data size: 460 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: decimal(34,16)), _col1 (type: decimal(34,16)), _col2 (type: binary)
- Reducer 7
- Execution mode: vectorized, llap
- Reduce Operator Tree:
- Group By Operator
- aggregations: min(VALUE._col0), max(VALUE._col1), bloom_filter(VALUE._col2, expectedEntries=1)
- mode: final
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 136 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- null sort order:
- sort order:
- Statistics: Num rows: 1 Data size: 136 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: tinyint), _col1 (type: tinyint), _col2 (type: binary)
+ Statistics: Num rows: 1 Data size: 516 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: decimal(34,16)), _col1 (type: decimal(34,16)), _col2 (type: tinyint), _col3 (type: tinyint), _col4 (type: bigint), _col5 (type: bigint), _col6 (type: binary)
Reducer 8
+ Execution mode: llap
+ Reduce Operator Tree:
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: decimal(26,12))
+ 1 _col0 (type: decimal(26,12))
+ outputColumnNames: _col1
+ Statistics: Num rows: 1 Data size: 167 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col1 (type: timestamp), -92 (type: int)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 167 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ keys: _col0 (type: timestamp), _col1 (type: int)
+ minReductionHashAggr: 0.99
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 167 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: timestamp), _col1 (type: int)
+ null sort order: zz
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: timestamp), _col1 (type: int)
+ Statistics: Num rows: 1 Data size: 167 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: timestamp)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 167 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: min(_col0), max(_col0), bloom_filter(_col0, expectedEntries=1)
+ minReductionHashAggr: 0.99
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 120 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 120 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: timestamp), _col1 (type: timestamp), _col2 (type: binary)
+ Reducer 9
Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: min(VALUE._col0), max(VALUE._col1), bloom_filter(VALUE._col2, expectedEntries=1)
mode: final
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 148 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 120 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
null sort order:
sort order:
- Statistics: Num rows: 1 Data size: 148 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint), _col1 (type: bigint), _col2 (type: binary)
+ Statistics: Num rows: 1 Data size: 120 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: timestamp), _col1 (type: timestamp), _col2 (type: binary)
Stage: Stage-0
Fetch Operator
diff --git a/ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_multicol.q.out b/ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_multicol.q.out
new file mode 100644
index 0000000..39db634
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_multicol.q.out
@@ -0,0 +1,808 @@
+PREHOOK: query: use tpch_0_001
+PREHOOK: type: SWITCHDATABASE
+PREHOOK: Input: database:tpch_0_001
+POSTHOOK: query: use tpch_0_001
+POSTHOOK: type: SWITCHDATABASE
+POSTHOOK: Input: database:tpch_0_001
+PREHOOK: query: explain
+select li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+from lineitem li
+ inner join partsupp ps on li.l_partkey = ps.ps_partkey and li.l_suppkey = ps.ps_suppkey
+where ps.ps_availqty < 15
+order by li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+PREHOOK: type: QUERY
+PREHOOK: Input: tpch_0_001@lineitem
+PREHOOK: Input: tpch_0_001@partsupp
+#### A masked pattern was here ####
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: li
+ filterExpr: (l_partkey is not null and l_suppkey is not null) (type: boolean)
+ Statistics: Num rows: 6005 Data size: 72060 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: (l_partkey is not null and l_suppkey is not null) (type: boolean)
+ Statistics: Num rows: 6005 Data size: 72060 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: l_orderkey (type: int), l_partkey (type: int), l_suppkey (type: int)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 6005 Data size: 72060 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col1 (type: int), _col2 (type: int)
+ null sort order: zz
+ sort order: ++
+ Map-reduce partition columns: _col1 (type: int), _col2 (type: int)
+ Statistics: Num rows: 6005 Data size: 72060 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: int)
+ Execution mode: vectorized, llap
+ LLAP IO: all inputs
+ Map 4
+ Map Operator Tree:
+ TableScan
+ alias: ps
+ filterExpr: (ps_availqty < 15) (type: boolean)
+ Statistics: Num rows: 800 Data size: 9600 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: (ps_availqty < 15) (type: boolean)
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: ps_partkey (type: int), ps_suppkey (type: int), ps_availqty (type: int)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: int)
+ null sort order: zz
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: int), _col1 (type: int)
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col2 (type: int)
+ Execution mode: vectorized, llap
+ LLAP IO: all inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col1 (type: int), _col2 (type: int)
+ 1 _col0 (type: int), _col1 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col5
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col5 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int)
+ null sort order: zzzz
+ sort order: ++++
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ Reducer 3
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: int), KEY.reducesinkkey1 (type: int), KEY.reducesinkkey2 (type: int), KEY.reducesinkkey3 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+from lineitem li
+ inner join partsupp ps on li.l_partkey = ps.ps_partkey and li.l_suppkey = ps.ps_suppkey
+where ps.ps_availqty < 15
+order by li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+PREHOOK: type: QUERY
+PREHOOK: Input: tpch_0_001@lineitem
+PREHOOK: Input: tpch_0_001@partsupp
+#### A masked pattern was here ####
+Stage-1 FILE SYSTEM COUNTERS:
+Stage-1 HIVE COUNTERS:
+ CREATED_FILES: 1
+ DESERIALIZE_ERRORS: 0
+ RECORDS_IN_Map_1: 6005
+ RECORDS_IN_Map_4: 800
+ RECORDS_OUT_0: 14
+ RECORDS_OUT_INTERMEDIATE_Map_1: 6005
+ RECORDS_OUT_INTERMEDIATE_Map_4: 2
+ RECORDS_OUT_INTERMEDIATE_Reducer_2: 14
+ RECORDS_OUT_INTERMEDIATE_Reducer_3: 0
+ RECORDS_OUT_OPERATOR_FIL_18: 6005
+ RECORDS_OUT_OPERATOR_FIL_21: 2
+ RECORDS_OUT_OPERATOR_FS_25: 14
+ RECORDS_OUT_OPERATOR_MAP_0: 0
+ RECORDS_OUT_OPERATOR_MERGEJOIN_17: 14
+ RECORDS_OUT_OPERATOR_RS_10: 14
+ RECORDS_OUT_OPERATOR_RS_20: 6005
+ RECORDS_OUT_OPERATOR_RS_23: 2
+ RECORDS_OUT_OPERATOR_SEL_19: 6005
+ RECORDS_OUT_OPERATOR_SEL_22: 2
+ RECORDS_OUT_OPERATOR_SEL_24: 14
+ RECORDS_OUT_OPERATOR_SEL_9: 14
+ RECORDS_OUT_OPERATOR_TS_0: 6005
+ RECORDS_OUT_OPERATOR_TS_3: 800
+ TOTAL_TABLE_ROWS_WRITTEN: 0
+Stage-1 LLAP IO COUNTERS:
+ CACHE_MISS_BYTES: 138342
+ NUM_DECODED_BATCHES: 2
+ NUM_VECTOR_BATCHES: 7
+ ROWS_EMITTED: 6805
+Stage-1 INPUT COUNTERS:
+ GROUPED_INPUT_SPLITS_Map_1: 1
+ GROUPED_INPUT_SPLITS_Map_4: 1
+ INPUT_DIRECTORIES_Map_1: 1
+ INPUT_DIRECTORIES_Map_4: 1
+ INPUT_FILES_Map_1: 1
+ INPUT_FILES_Map_4: 1
+ RAW_INPUT_SPLITS_Map_1: 1
+ RAW_INPUT_SPLITS_Map_4: 1
+134 145 6 11
+192 142 9 13
+583 145 6 11
+998 142 9 13
+2081 142 9 13
+2306 142 9 13
+2307 142 9 13
+3554 145 6 11
+3556 142 9 13
+4711 145 6 11
+4807 145 6 11
+5158 142 9 13
+5413 142 9 13
+5670 142 9 13
+PREHOOK: query: explain
+select li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+from lineitem li
+ inner join partsupp ps on li.l_partkey = ps.ps_partkey and li.l_suppkey = ps.ps_suppkey
+where ps.ps_availqty < 15
+order by li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+PREHOOK: type: QUERY
+PREHOOK: Input: tpch_0_001@lineitem
+PREHOOK: Input: tpch_0_001@partsupp
+#### A masked pattern was here ####
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Map 1 <- Reducer 5 (BROADCAST_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+ Reducer 5 <- Map 4 (CUSTOM_SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: li
+ filterExpr: (l_partkey is not null and l_suppkey is not null and l_partkey BETWEEN DynamicValue(RS_7_ps_ps_partkey_min) AND DynamicValue(RS_7_ps_ps_partkey_max) and in_bloom_filter(l_partkey, DynamicValue(RS_7_ps_ps_partkey_bloom_filter))) (type: boolean)
+ Statistics: Num rows: 6005 Data size: 72060 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: (l_partkey is not null and l_suppkey is not null and l_partkey BETWEEN DynamicValue(RS_7_ps_ps_partkey_min) AND DynamicValue(RS_7_ps_ps_partkey_max) and in_bloom_filter(l_partkey, DynamicValue(RS_7_ps_ps_partkey_bloom_filter))) (type: boolean)
+ Statistics: Num rows: 6005 Data size: 72060 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: l_orderkey (type: int), l_partkey (type: int), l_suppkey (type: int)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 6005 Data size: 72060 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col1 (type: int), _col2 (type: int)
+ null sort order: zz
+ sort order: ++
+ Map-reduce partition columns: _col1 (type: int), _col2 (type: int)
+ Statistics: Num rows: 6005 Data size: 72060 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: int)
+ Execution mode: vectorized, llap
+ LLAP IO: all inputs
+ Map 4
+ Map Operator Tree:
+ TableScan
+ alias: ps
+ filterExpr: (ps_availqty < 15) (type: boolean)
+ Statistics: Num rows: 800 Data size: 9600 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: (ps_availqty < 15) (type: boolean)
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: ps_partkey (type: int), ps_suppkey (type: int), ps_availqty (type: int)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: int)
+ null sort order: zz
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: int), _col1 (type: int)
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col2 (type: int)
+ Select Operator
+ expressions: _col0 (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ aggregations: min(_col0), max(_col0), bloom_filter(_col0, expectedEntries=1000000)
+ minReductionHashAggr: 0.0
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: int), _col1 (type: int), _col2 (type: binary)
+ Execution mode: vectorized, llap
+ LLAP IO: all inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col1 (type: int), _col2 (type: int)
+ 1 _col0 (type: int), _col1 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col5
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col5 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int)
+ null sort order: zzzz
+ sort order: ++++
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ Reducer 3
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: int), KEY.reducesinkkey1 (type: int), KEY.reducesinkkey2 (type: int), KEY.reducesinkkey3 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Reducer 5
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: min(VALUE._col0), max(VALUE._col1), bloom_filter(VALUE._col2, expectedEntries=1000000)
+ mode: final
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: int), _col1 (type: int), _col2 (type: binary)
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+from lineitem li
+ inner join partsupp ps on li.l_partkey = ps.ps_partkey and li.l_suppkey = ps.ps_suppkey
+where ps.ps_availqty < 15
+order by li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+PREHOOK: type: QUERY
+PREHOOK: Input: tpch_0_001@lineitem
+PREHOOK: Input: tpch_0_001@partsupp
+#### A masked pattern was here ####
+Stage-1 FILE SYSTEM COUNTERS:
+Stage-1 HIVE COUNTERS:
+ CREATED_FILES: 1
+ DESERIALIZE_ERRORS: 0
+ RECORDS_IN_Map_1: 6005
+ RECORDS_IN_Map_4: 800
+ RECORDS_OUT_0: 14
+ RECORDS_OUT_INTERMEDIATE_Map_1: 50
+ RECORDS_OUT_INTERMEDIATE_Map_4: 3
+ RECORDS_OUT_INTERMEDIATE_Reducer_2: 14
+ RECORDS_OUT_INTERMEDIATE_Reducer_3: 0
+ RECORDS_OUT_INTERMEDIATE_Reducer_5: 1
+ RECORDS_OUT_OPERATOR_FIL_38: 2
+ RECORDS_OUT_OPERATOR_FIL_46: 50
+ RECORDS_OUT_OPERATOR_FS_50: 14
+ RECORDS_OUT_OPERATOR_GBY_42: 1
+ RECORDS_OUT_OPERATOR_GBY_44: 1
+ RECORDS_OUT_OPERATOR_MAP_0: 0
+ RECORDS_OUT_OPERATOR_MERGEJOIN_37: 14
+ RECORDS_OUT_OPERATOR_RS_10: 14
+ RECORDS_OUT_OPERATOR_RS_40: 2
+ RECORDS_OUT_OPERATOR_RS_43: 1
+ RECORDS_OUT_OPERATOR_RS_45: 1
+ RECORDS_OUT_OPERATOR_RS_48: 50
+ RECORDS_OUT_OPERATOR_SEL_39: 2
+ RECORDS_OUT_OPERATOR_SEL_41: 2
+ RECORDS_OUT_OPERATOR_SEL_47: 50
+ RECORDS_OUT_OPERATOR_SEL_49: 14
+ RECORDS_OUT_OPERATOR_SEL_9: 14
+ RECORDS_OUT_OPERATOR_TS_0: 6005
+ RECORDS_OUT_OPERATOR_TS_3: 800
+ TOTAL_TABLE_ROWS_WRITTEN: 0
+Stage-1 LLAP IO COUNTERS:
+ CACHE_HIT_BYTES: 138344
+ NUM_DECODED_BATCHES: 2
+ NUM_VECTOR_BATCHES: 7
+ ROWS_EMITTED: 6805
+Stage-1 INPUT COUNTERS:
+ GROUPED_INPUT_SPLITS_Map_1: 1
+ GROUPED_INPUT_SPLITS_Map_4: 1
+ INPUT_DIRECTORIES_Map_1: 1
+ INPUT_DIRECTORIES_Map_4: 1
+ INPUT_FILES_Map_1: 1
+ INPUT_FILES_Map_4: 1
+ RAW_INPUT_SPLITS_Map_1: 1
+ RAW_INPUT_SPLITS_Map_4: 1
+134 145 6 11
+192 142 9 13
+583 145 6 11
+998 142 9 13
+2081 142 9 13
+2306 142 9 13
+2307 142 9 13
+3554 145 6 11
+3556 142 9 13
+4711 145 6 11
+4807 145 6 11
+5158 142 9 13
+5413 142 9 13
+5670 142 9 13
+PREHOOK: query: explain
+select li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+from lineitem li
+ inner join partsupp ps on li.l_partkey = ps.ps_partkey and li.l_suppkey = ps.ps_suppkey
+where ps.ps_availqty < 15
+order by li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+PREHOOK: type: QUERY
+PREHOOK: Input: tpch_0_001@lineitem
+PREHOOK: Input: tpch_0_001@partsupp
+#### A masked pattern was here ####
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Map 1 <- Reducer 5 (BROADCAST_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+ Reducer 5 <- Map 4 (CUSTOM_SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: li
+ filterExpr: (l_partkey is not null and l_suppkey is not null and l_partkey BETWEEN DynamicValue(RS[41]_col0) AND DynamicValue(RS[41]_col1) and l_suppkey BETWEEN DynamicValue(RS[41]_col2) AND DynamicValue(RS[41]_col3) and in_bloom_filter(hash(l_partkey,l_suppkey), DynamicValue(RS[41]_col4))) (type: boolean)
+ Statistics: Num rows: 6005 Data size: 72060 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: (l_partkey is not null and l_suppkey is not null and l_partkey BETWEEN DynamicValue(RS[41]_col0) AND DynamicValue(RS[41]_col1) and l_suppkey BETWEEN DynamicValue(RS[41]_col2) AND DynamicValue(RS[41]_col3) and in_bloom_filter(hash(l_partkey,l_suppkey), DynamicValue(RS[41]_col4))) (type: boolean)
+ Statistics: Num rows: 6005 Data size: 72060 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: l_orderkey (type: int), l_partkey (type: int), l_suppkey (type: int)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 6005 Data size: 72060 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col1 (type: int), _col2 (type: int)
+ null sort order: zz
+ sort order: ++
+ Map-reduce partition columns: _col1 (type: int), _col2 (type: int)
+ Statistics: Num rows: 6005 Data size: 72060 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: int)
+ Execution mode: vectorized, llap
+ LLAP IO: all inputs
+ Map 4
+ Map Operator Tree:
+ TableScan
+ alias: ps
+ filterExpr: (ps_availqty < 15) (type: boolean)
+ Statistics: Num rows: 800 Data size: 9600 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: (ps_availqty < 15) (type: boolean)
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: ps_partkey (type: int), ps_suppkey (type: int), ps_availqty (type: int)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: int)
+ null sort order: zz
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: int), _col1 (type: int)
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col2 (type: int)
+ Select Operator
+ expressions: _col0 (type: int), _col1 (type: int), hash(_col0,_col1) (type: int)
+ outputColumnNames: _col0, _col1, _col3
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ aggregations: min(_col0), max(_col0), min(_col1), max(_col1), bloom_filter(_col3, expectedEntries=1000000)
+ minReductionHashAggr: 0.0
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4
+ Statistics: Num rows: 1 Data size: 160 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 160 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: binary)
+ Execution mode: vectorized, llap
+ LLAP IO: all inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col1 (type: int), _col2 (type: int)
+ 1 _col0 (type: int), _col1 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col5
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col5 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int)
+ null sort order: zzzz
+ sort order: ++++
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ Reducer 3
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: int), KEY.reducesinkkey1 (type: int), KEY.reducesinkkey2 (type: int), KEY.reducesinkkey3 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Reducer 5
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: min(VALUE._col0), max(VALUE._col1), min(VALUE._col2), max(VALUE._col3), bloom_filter(VALUE._col4, expectedEntries=1000000)
+ mode: final
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4
+ Statistics: Num rows: 1 Data size: 160 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 160 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: binary)
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+from lineitem li
+ inner join partsupp ps on li.l_partkey = ps.ps_partkey and li.l_suppkey = ps.ps_suppkey
+where ps.ps_availqty < 15
+order by li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+PREHOOK: type: QUERY
+PREHOOK: Input: tpch_0_001@lineitem
+PREHOOK: Input: tpch_0_001@partsupp
+#### A masked pattern was here ####
+Stage-1 FILE SYSTEM COUNTERS:
+Stage-1 HIVE COUNTERS:
+ CREATED_FILES: 1
+ DESERIALIZE_ERRORS: 0
+ RECORDS_IN_Map_1: 6005
+ RECORDS_IN_Map_4: 800
+ RECORDS_OUT_0: 14
+ RECORDS_OUT_INTERMEDIATE_Map_1: 14
+ RECORDS_OUT_INTERMEDIATE_Map_4: 3
+ RECORDS_OUT_INTERMEDIATE_Reducer_2: 14
+ RECORDS_OUT_INTERMEDIATE_Reducer_3: 0
+ RECORDS_OUT_INTERMEDIATE_Reducer_5: 1
+ RECORDS_OUT_OPERATOR_FIL_48: 2
+ RECORDS_OUT_OPERATOR_FIL_56: 14
+ RECORDS_OUT_OPERATOR_FS_60: 14
+ RECORDS_OUT_OPERATOR_GBY_52: 1
+ RECORDS_OUT_OPERATOR_GBY_54: 1
+ RECORDS_OUT_OPERATOR_MAP_0: 0
+ RECORDS_OUT_OPERATOR_MERGEJOIN_47: 14
+ RECORDS_OUT_OPERATOR_RS_10: 14
+ RECORDS_OUT_OPERATOR_RS_50: 2
+ RECORDS_OUT_OPERATOR_RS_53: 1
+ RECORDS_OUT_OPERATOR_RS_55: 1
+ RECORDS_OUT_OPERATOR_RS_58: 14
+ RECORDS_OUT_OPERATOR_SEL_49: 2
+ RECORDS_OUT_OPERATOR_SEL_51: 2
+ RECORDS_OUT_OPERATOR_SEL_57: 14
+ RECORDS_OUT_OPERATOR_SEL_59: 14
+ RECORDS_OUT_OPERATOR_SEL_9: 14
+ RECORDS_OUT_OPERATOR_TS_0: 6005
+ RECORDS_OUT_OPERATOR_TS_3: 800
+ TOTAL_TABLE_ROWS_WRITTEN: 0
+Stage-1 LLAP IO COUNTERS:
+ CACHE_HIT_BYTES: 138344
+ NUM_DECODED_BATCHES: 2
+ NUM_VECTOR_BATCHES: 7
+ ROWS_EMITTED: 6805
+Stage-1 INPUT COUNTERS:
+ GROUPED_INPUT_SPLITS_Map_1: 1
+ GROUPED_INPUT_SPLITS_Map_4: 1
+ INPUT_DIRECTORIES_Map_1: 1
+ INPUT_DIRECTORIES_Map_4: 1
+ INPUT_FILES_Map_1: 1
+ INPUT_FILES_Map_4: 1
+ RAW_INPUT_SPLITS_Map_1: 1
+ RAW_INPUT_SPLITS_Map_4: 1
+134 145 6 11
+192 142 9 13
+583 145 6 11
+998 142 9 13
+2081 142 9 13
+2306 142 9 13
+2307 142 9 13
+3554 145 6 11
+3556 142 9 13
+4711 145 6 11
+4807 145 6 11
+5158 142 9 13
+5413 142 9 13
+5670 142 9 13
+PREHOOK: query: explain
+select
+ /*+ semi(ps, ps_partkey, li, 2), semi(ps, ps_suppkey, li, 3)*/
+ li.l_orderkey,
+ li.l_partkey,
+ li.l_suppkey,
+ ps.ps_availqty
+from lineitem li
+ inner join partsupp ps on li.l_partkey = ps.ps_partkey and li.l_suppkey = ps.ps_suppkey
+where ps.ps_availqty < 15
+order by li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+PREHOOK: type: QUERY
+PREHOOK: Input: tpch_0_001@lineitem
+PREHOOK: Input: tpch_0_001@partsupp
+#### A masked pattern was here ####
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Map 1 <- Reducer 5 (BROADCAST_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+ Reducer 5 <- Map 4 (CUSTOM_SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: li
+ filterExpr: (l_partkey is not null and l_suppkey is not null and l_partkey BETWEEN DynamicValue(RS[31]_col0) AND DynamicValue(RS[31]_col1) and l_suppkey BETWEEN DynamicValue(RS[31]_col2) AND DynamicValue(RS[31]_col3) and in_bloom_filter(hash(l_partkey,l_suppkey), DynamicValue(RS[31]_col4))) (type: boolean)
+ Statistics: Num rows: 6005 Data size: 72060 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: (l_partkey is not null and l_suppkey is not null and l_partkey BETWEEN DynamicValue(RS[31]_col0) AND DynamicValue(RS[31]_col1) and l_suppkey BETWEEN DynamicValue(RS[31]_col2) AND DynamicValue(RS[31]_col3) and in_bloom_filter(hash(l_partkey,l_suppkey), DynamicValue(RS[31]_col4))) (type: boolean)
+ Statistics: Num rows: 6005 Data size: 72060 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: l_orderkey (type: int), l_partkey (type: int), l_suppkey (type: int)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 6005 Data size: 72060 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col1 (type: int), _col2 (type: int)
+ null sort order: zz
+ sort order: ++
+ Map-reduce partition columns: _col1 (type: int), _col2 (type: int)
+ Statistics: Num rows: 6005 Data size: 72060 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: int)
+ Execution mode: vectorized, llap
+ LLAP IO: all inputs
+ Map 4
+ Map Operator Tree:
+ TableScan
+ alias: ps
+ filterExpr: (ps_availqty < 15) (type: boolean)
+ Statistics: Num rows: 800 Data size: 9600 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: (ps_availqty < 15) (type: boolean)
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: ps_partkey (type: int), ps_suppkey (type: int), ps_availqty (type: int)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: int)
+ null sort order: zz
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: int), _col1 (type: int)
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col2 (type: int)
+ Select Operator
+ expressions: _col0 (type: int), _col1 (type: int), hash(_col0,_col1) (type: int)
+ outputColumnNames: _col0, _col1, _col3
+ Statistics: Num rows: 1 Data size: 12 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ aggregations: min(_col0), max(_col0), min(_col1), max(_col1), bloom_filter(_col3, expectedEntries=3)
+ minReductionHashAggr: 0.0
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4
+ Statistics: Num rows: 1 Data size: 160 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 160 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: binary)
+ Execution mode: vectorized, llap
+ LLAP IO: all inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col1 (type: int), _col2 (type: int)
+ 1 _col0 (type: int), _col1 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col5
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col5 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int)
+ null sort order: zzzz
+ sort order: ++++
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ Reducer 3
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: int), KEY.reducesinkkey1 (type: int), KEY.reducesinkkey2 (type: int), KEY.reducesinkkey3 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 30 Data size: 480 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Reducer 5
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: min(VALUE._col0), max(VALUE._col1), min(VALUE._col2), max(VALUE._col3), bloom_filter(VALUE._col4, expectedEntries=3)
+ mode: final
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4
+ Statistics: Num rows: 1 Data size: 160 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 160 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: binary)
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select
+ /*+ semi(ps, ps_partkey, li, 2), semi(ps, ps_suppkey, li, 3)*/
+ li.l_orderkey,
+ li.l_partkey,
+ li.l_suppkey,
+ ps.ps_availqty
+from lineitem li
+ inner join partsupp ps on li.l_partkey = ps.ps_partkey and li.l_suppkey = ps.ps_suppkey
+where ps.ps_availqty < 15
+order by li.l_orderkey, li.l_partkey, li.l_suppkey, ps.ps_availqty
+PREHOOK: type: QUERY
+PREHOOK: Input: tpch_0_001@lineitem
+PREHOOK: Input: tpch_0_001@partsupp
+#### A masked pattern was here ####
+Stage-1 FILE SYSTEM COUNTERS:
+Stage-1 HIVE COUNTERS:
+ CREATED_FILES: 1
+ DESERIALIZE_ERRORS: 0
+ RECORDS_IN_Map_1: 6005
+ RECORDS_IN_Map_4: 800
+ RECORDS_OUT_0: 14
+ RECORDS_OUT_INTERMEDIATE_Map_1: 14
+ RECORDS_OUT_INTERMEDIATE_Map_4: 3
+ RECORDS_OUT_INTERMEDIATE_Reducer_2: 14
+ RECORDS_OUT_INTERMEDIATE_Reducer_3: 0
+ RECORDS_OUT_INTERMEDIATE_Reducer_5: 1
+ RECORDS_OUT_OPERATOR_FIL_33: 2
+ RECORDS_OUT_OPERATOR_FIL_41: 14
+ RECORDS_OUT_OPERATOR_FS_45: 14
+ RECORDS_OUT_OPERATOR_GBY_37: 1
+ RECORDS_OUT_OPERATOR_GBY_39: 1
+ RECORDS_OUT_OPERATOR_MAP_0: 0
+ RECORDS_OUT_OPERATOR_MERGEJOIN_32: 14
+ RECORDS_OUT_OPERATOR_RS_10: 14
+ RECORDS_OUT_OPERATOR_RS_35: 2
+ RECORDS_OUT_OPERATOR_RS_38: 1
+ RECORDS_OUT_OPERATOR_RS_40: 1
+ RECORDS_OUT_OPERATOR_RS_43: 14
+ RECORDS_OUT_OPERATOR_SEL_34: 2
+ RECORDS_OUT_OPERATOR_SEL_36: 2
+ RECORDS_OUT_OPERATOR_SEL_42: 14
+ RECORDS_OUT_OPERATOR_SEL_44: 14
+ RECORDS_OUT_OPERATOR_SEL_9: 14
+ RECORDS_OUT_OPERATOR_TS_0: 6005
+ RECORDS_OUT_OPERATOR_TS_3: 800
+ TOTAL_TABLE_ROWS_WRITTEN: 0
+Stage-1 LLAP IO COUNTERS:
+ CACHE_HIT_BYTES: 138344
+ NUM_DECODED_BATCHES: 2
+ NUM_VECTOR_BATCHES: 7
+ ROWS_EMITTED: 6805
+Stage-1 INPUT COUNTERS:
+ GROUPED_INPUT_SPLITS_Map_1: 1
+ GROUPED_INPUT_SPLITS_Map_4: 1
+ INPUT_DIRECTORIES_Map_1: 1
+ INPUT_DIRECTORIES_Map_4: 1
+ INPUT_FILES_Map_1: 1
+ INPUT_FILES_Map_4: 1
+ RAW_INPUT_SPLITS_Map_1: 1
+ RAW_INPUT_SPLITS_Map_4: 1
+134 145 6 11
+192 142 9 13
+583 145 6 11
+998 142 9 13
+2081 142 9 13
+2306 142 9 13
+2307 142 9 13
+3554 145 6 11
+3556 142 9 13
+4711 145 6 11
+4807 145 6 11
+5158 142 9 13
+5413 142 9 13
+5670 142 9 13
diff --git a/ql/src/test/results/clientpositive/llap/dynamic_semijoin_user_level.q.out b/ql/src/test/results/clientpositive/llap/dynamic_semijoin_user_level.q.out
index 9bce543..bf8ba21 100644
--- a/ql/src/test/results/clientpositive/llap/dynamic_semijoin_user_level.q.out
+++ b/ql/src/test/results/clientpositive/llap/dynamic_semijoin_user_level.q.out
@@ -638,7 +638,7 @@ Stage-0
PARTITION_ONLY_SHUFFLE [RS_11]
Group By Operator [GBY_10] (rows=1 width=8)
Output:["_col0"],aggregations:["count()"]
- Merge Join Operator [MERGEJOIN_39] (rows=2200 width=178)
+ Merge Join Operator [MERGEJOIN_49] (rows=2200 width=178)
Conds:RS_6._col0, _col1=RS_7._col0, _col1(Inner)
<-Map 4 [SIMPLE_EDGE] llap
PARTITION_ONLY_SHUFFLE [RS_7]
@@ -655,19 +655,19 @@ Stage-0
Select Operator [SEL_2] (rows=2000 width=178)
Output:["_col0","_col1"]
Filter Operator [FIL_17] (rows=2000 width=178)
- predicate:(key is not null and value is not null and key BETWEEN DynamicValue(RS_7_srcpart_small_n4_key1_min) AND DynamicValue(RS_7_srcpart_small_n4_key1_max) and in_bloom_filter(key, DynamicValue(RS_7_srcpart_small_n4_key1_bloom_filter)))
+ predicate:(key is not null and value is not null and key BETWEEN DynamicValue(RS[43]_col0) AND DynamicValue(RS[43]_col1) and value BETWEEN DynamicValue(RS[43]_col2) AND DynamicValue(RS[43]_col3) and in_bloom_filter(hash(key,value), DynamicValue(RS[43]_col4)))
TableScan [TS_0] (rows=2000 width=178)
default@srcpart_date_n9,srcpart_date_n9,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
<-Reducer 5 [BROADCAST_EDGE] llap
- BROADCAST [RS_23]
- Group By Operator [GBY_22] (rows=1 width=730)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=20)"]
+ BROADCAST [RS_43]
+ Group By Operator [GBY_42] (rows=1 width=1058)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","min(VALUE._col2)","max(VALUE._col3)","bloom_filter(VALUE._col4, expectedEntries=20)"]
<-Map 4 [CUSTOM_SIMPLE_EDGE] llap
- PARTITION_ONLY_SHUFFLE [RS_21]
- Group By Operator [GBY_20] (rows=1 width=730)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=20)"]
- Select Operator [SEL_19] (rows=20 width=178)
- Output:["_col0"]
+ PARTITION_ONLY_SHUFFLE [RS_41]
+ Group By Operator [GBY_40] (rows=1 width=1058)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["min(_col0)","max(_col0)","min(_col1)","max(_col1)","bloom_filter(_col3, expectedEntries=20)"]
+ Select Operator [SEL_39] (rows=20 width=178)
+ Output:["_col0","_col1","_col3"]
Please refer to the previous Select Operator [SEL_5]
PREHOOK: query: select count(*) from srcpart_date_n9 join srcpart_small_n4 on (srcpart_date_n9.key = srcpart_small_n4.key1 and srcpart_date_n9.value = srcpart_small_n4.value1)
diff --git a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out b/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
index cce2f97..4750359 100644
--- a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
+++ b/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
@@ -2154,7 +2154,7 @@ Stage-0
Stage-1
Reducer 3 llap
File Output Operator [FS_15]
- Merge Join Operator [MERGEJOIN_28] (rows=167 width=178)
+ Merge Join Operator [MERGEJOIN_33] (rows=167 width=178)
Conds:RS_11._col0, _col1=RS_12._col0, _col1(Anti),Output:["_col0","_col1"]
<-Map 1 [SIMPLE_EDGE] llap
SHUFFLE [RS_12]
@@ -2226,7 +2226,7 @@ Stage-0
Stage-1
Reducer 2 llap
File Output Operator [FS_12]
- Merge Join Operator [MERGEJOIN_37] (rows=83 width=178)
+ Merge Join Operator [MERGEJOIN_47] (rows=83 width=178)
Conds:RS_8._col0, _col1=RS_9._col0, _col1(Left Semi),Output:["_col0","_col1"]
<-Map 1 [SIMPLE_EDGE] llap
SHUFFLE [RS_8]
@@ -2282,7 +2282,7 @@ Stage-0
Stage-1
Reducer 2 llap
File Output Operator [FS_12]
- Merge Join Operator [MERGEJOIN_37] (rows=83 width=178)
+ Merge Join Operator [MERGEJOIN_47] (rows=83 width=178)
Conds:RS_8._col0, _col1=RS_9._col0, _col1(Left Semi),Output:["_col0","_col1"]
<-Map 1 [SIMPLE_EDGE] llap
SHUFFLE [RS_8]
@@ -2696,12 +2696,12 @@ Stage-0
Output:["_col0","_col1","_col2"]
Filter Operator [FIL_22] (rows=38 width=228)
predicate:(_col4 is null or (_col4 = 0L) or (_col7 is not null or _col0 is null or (_col5 < _col4)) is not true)
- Merge Join Operator [MERGEJOIN_45] (rows=38 width=228)
+ Merge Join Operator [MERGEJOIN_50] (rows=38 width=228)
Conds:RS_19._col0, _col1=RS_20._col0, _col2(Left Outer),Output:["_col0","_col1","_col2","_col4","_col5","_col7"]
<-Reducer 2 [SIMPLE_EDGE] llap
SHUFFLE [RS_19]
PartitionCols:_col0, _col1
- Merge Join Operator [MERGEJOIN_44] (rows=36 width=227)
+ Merge Join Operator [MERGEJOIN_49] (rows=36 width=227)
Conds:RS_16._col1=RS_17._col0(Left Outer),Output:["_col0","_col1","_col2","_col4","_col5"]
<-Map 1 [SIMPLE_EDGE] llap
SHUFFLE [RS_16]
@@ -6802,7 +6802,7 @@ Stage-0
Output:["_col0","_col1"]
Filter Operator [FIL_18] (rows=500 width=179)
predicate:_col2 is null
- Map Join Operator [MAPJOIN_33] (rows=762 width=179)
+ Map Join Operator [MAPJOIN_38] (rows=762 width=179)
Conds:GBY_5._col0, _col1=RS_16._col1, _col2(Left Outer),Output:["_col0","_col1","_col2"]
<-Reducer 5 [BROADCAST_EDGE] llap
BROADCAST [RS_16]
@@ -6888,7 +6888,7 @@ Stage-0
Output:["_col0","_col1"]
Filter Operator [FIL_13] (rows=500 width=179)
predicate:_col2 is null
- Map Join Operator [MAPJOIN_28] (rows=762 width=179)
+ Map Join Operator [MAPJOIN_33] (rows=762 width=179)
Conds:SEL_1._col0, _col1=RS_11._col1, _col2(Left Outer),Output:["_col0","_col1","_col2"]
<-Reducer 4 [BROADCAST_EDGE] llap
BROADCAST [RS_11]
@@ -6953,7 +6953,7 @@ Stage-0
Output:["_col0","_col1"]
Filter Operator [FIL_13] (rows=500 width=179)
predicate:_col2 is null
- Map Join Operator [MAPJOIN_28] (rows=762 width=179)
+ Map Join Operator [MAPJOIN_33] (rows=762 width=179)
Conds:SEL_1._col0, _col1=RS_11._col1, _col2(Left Outer),Output:["_col0","_col1","_col2"]
<-Reducer 4 [BROADCAST_EDGE] llap
BROADCAST [RS_11]
diff --git a/ql/src/test/results/clientpositive/llap/explainuser_2.q.out b/ql/src/test/results/clientpositive/llap/explainuser_2.q.out
index af31e07..4cb4f71 100644
--- a/ql/src/test/results/clientpositive/llap/explainuser_2.q.out
+++ b/ql/src/test/results/clientpositive/llap/explainuser_2.q.out
@@ -335,14 +335,14 @@ Stage-0
limit:100
Stage-1
Reducer 5 vectorized, llap
- File Output Operator [FS_222]
- Limit [LIM_221] (rows=2 width=285)
+ File Output Operator [FS_242]
+ Limit [LIM_241] (rows=2 width=285)
Number of rows:100
- Select Operator [SEL_220] (rows=2 width=285)
+ Select Operator [SEL_240] (rows=2 width=285)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
<-Reducer 4 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_219]
- Group By Operator [GBY_218] (rows=2 width=285)
+ SHUFFLE [RS_239]
+ Group By Operator [GBY_238] (rows=2 width=285)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["count(VALUE._col0)","count(VALUE._col1)","count(VALUE._col2)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 3 [SIMPLE_EDGE] llap
SHUFFLE [RS_49]
@@ -351,107 +351,107 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["count(_col8)","count(_col15)","count(_col3)"],keys:_col7, _col14, _col2
Top N Key Operator [TNK_94] (rows=28 width=534)
keys:_col7, _col14, _col2,top n:100
- Merge Join Operator [MERGEJOIN_193] (rows=28 width=534)
+ Merge Join Operator [MERGEJOIN_213] (rows=28 width=534)
Conds:RS_44._col1, _col3=RS_45._col10, _col12(Inner),Output:["_col2","_col3","_col7","_col8","_col14","_col15"]
<-Reducer 10 [SIMPLE_EDGE] llap
SHUFFLE [RS_45]
PartitionCols:_col10, _col12
Select Operator [SEL_40] (rows=2 width=447)
Output:["_col2","_col3","_col9","_col10","_col12"]
- Merge Join Operator [MERGEJOIN_192] (rows=2 width=447)
+ Merge Join Operator [MERGEJOIN_212] (rows=2 width=447)
Conds:RS_37._col2, _col4=RS_38._col1, _col3(Inner),Output:["_col0","_col1","_col9","_col10","_col12"]
<-Reducer 11 [SIMPLE_EDGE] llap
SHUFFLE [RS_38]
PartitionCols:_col1, _col3
- Merge Join Operator [MERGEJOIN_191] (rows=5 width=356)
- Conds:RS_217._col0=RS_205._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_211] (rows=5 width=356)
+ Conds:RS_237._col0=RS_225._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
<-Map 6 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_205]
+ SHUFFLE [RS_225]
PartitionCols:_col0
- Select Operator [SEL_202] (rows=5 width=87)
+ Select Operator [SEL_222] (rows=5 width=87)
Output:["_col0"]
- Filter Operator [FIL_199] (rows=5 width=178)
+ Filter Operator [FIL_219] (rows=5 width=178)
predicate:((value) IN ('2000Q1', '2000Q2', '2000Q3') and key is not null)
TableScan [TS_3] (rows=500 width=178)
default@src,d3,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
<-Map 15 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_217]
+ SHUFFLE [RS_237]
PartitionCols:_col0
- Select Operator [SEL_216] (rows=7 width=443)
+ Select Operator [SEL_236] (rows=7 width=443)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_215] (rows=7 width=534)
+ Filter Operator [FIL_235] (rows=7 width=534)
predicate:((v1 = 'srv1') and k2 is not null and k3 is not null and v2 is not null and v3 is not null and k1 is not null)
TableScan [TS_18] (rows=85 width=534)
default@sr,sr,Tbl:COMPLETE,Col:COMPLETE,Output:["k1","v1","k2","v2","k3","v3"]
<-Reducer 9 [SIMPLE_EDGE] llap
SHUFFLE [RS_37]
PartitionCols:_col2, _col4
- Merge Join Operator [MERGEJOIN_190] (rows=2 width=352)
- Conds:RS_34._col1=RS_214._col0(Inner),Output:["_col0","_col1","_col2","_col4"]
+ Merge Join Operator [MERGEJOIN_210] (rows=2 width=352)
+ Conds:RS_34._col1=RS_234._col0(Inner),Output:["_col0","_col1","_col2","_col4"]
<-Map 14 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_214]
+ SHUFFLE [RS_234]
PartitionCols:_col0
- Select Operator [SEL_213] (rows=2 width=89)
+ Select Operator [SEL_233] (rows=2 width=89)
Output:["_col0"]
- Filter Operator [FIL_212] (rows=2 width=175)
+ Filter Operator [FIL_232] (rows=2 width=175)
predicate:((key = 'src1key') and value is not null)
TableScan [TS_15] (rows=25 width=175)
default@src1,src1,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
<-Reducer 8 [SIMPLE_EDGE] llap
SHUFFLE [RS_34]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_189] (rows=2 width=352)
- Conds:RS_31._col3=RS_211._col0(Inner),Output:["_col0","_col1","_col2","_col4"]
+ Merge Join Operator [MERGEJOIN_209] (rows=2 width=352)
+ Conds:RS_31._col3=RS_231._col0(Inner),Output:["_col0","_col1","_col2","_col4"]
<-Map 13 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_211]
+ SHUFFLE [RS_231]
PartitionCols:_col0
- Select Operator [SEL_210] (rows=6 width=91)
+ Select Operator [SEL_230] (rows=6 width=91)
Output:["_col0"]
- Filter Operator [FIL_209] (rows=6 width=178)
+ Filter Operator [FIL_229] (rows=6 width=178)
predicate:((key = 'srcpartkey') and value is not null)
TableScan [TS_12] (rows=2000 width=178)
default@srcpart,srcpart,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
<-Reducer 7 [SIMPLE_EDGE] llap
SHUFFLE [RS_31]
PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_188] (rows=2 width=443)
- Conds:RS_208._col0=RS_204._col0(Inner),Output:["_col0","_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_208] (rows=2 width=443)
+ Conds:RS_228._col0=RS_224._col0(Inner),Output:["_col0","_col1","_col2","_col3","_col4"]
<-Map 6 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_204]
+ SHUFFLE [RS_224]
PartitionCols:_col0
- Select Operator [SEL_201] (rows=2 width=87)
+ Select Operator [SEL_221] (rows=2 width=87)
Output:["_col0"]
- Filter Operator [FIL_198] (rows=2 width=178)
+ Filter Operator [FIL_218] (rows=2 width=178)
predicate:((value = 'd1value') and key is not null)
Please refer to the previous TableScan [TS_3]
<-Map 12 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_208]
+ SHUFFLE [RS_228]
PartitionCols:_col0
- Select Operator [SEL_207] (rows=7 width=443)
+ Select Operator [SEL_227] (rows=7 width=443)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_206] (rows=7 width=534)
+ Filter Operator [FIL_226] (rows=7 width=534)
predicate:((v3 = 'ssv3') and k2 is not null and k3 is not null and k1 is not null and v1 is not null and v2 is not null)
TableScan [TS_6] (rows=85 width=534)
default@ss_n1,ss_n1,Tbl:COMPLETE,Col:COMPLETE,Output:["k1","v1","k2","v2","k3","v3"]
<-Reducer 2 [SIMPLE_EDGE] llap
SHUFFLE [RS_44]
PartitionCols:_col1, _col3
- Merge Join Operator [MERGEJOIN_187] (rows=70 width=269)
- Conds:RS_196._col0=RS_203._col0(Inner),Output:["_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_207] (rows=70 width=269)
+ Conds:RS_216._col0=RS_223._col0(Inner),Output:["_col1","_col2","_col3"]
<-Map 6 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_203]
+ SHUFFLE [RS_223]
PartitionCols:_col0
- Select Operator [SEL_200] (rows=5 width=87)
+ Select Operator [SEL_220] (rows=5 width=87)
Output:["_col0"]
- Filter Operator [FIL_197] (rows=5 width=178)
+ Filter Operator [FIL_217] (rows=5 width=178)
predicate:((value) IN ('2000Q1', '2000Q2', '2000Q3') and key is not null)
Please refer to the previous TableScan [TS_3]
<-Map 1 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_196]
+ SHUFFLE [RS_216]
PartitionCols:_col0
- Select Operator [SEL_195] (rows=170 width=356)
+ Select Operator [SEL_215] (rows=170 width=356)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_194] (rows=170 width=356)
+ Filter Operator [FIL_214] (rows=170 width=356)
predicate:(v2 is not null and v3 is not null and k1 is not null)
TableScan [TS_0] (rows=170 width=356)
default@cs,cs,Tbl:COMPLETE,Col:COMPLETE,Output:["k1","v2","k3","v3"]
@@ -1069,104 +1069,104 @@ Stage-0
limit:100
Stage-1
Reducer 5 vectorized, llap
- File Output Operator [FS_239]
- Limit [LIM_238] (rows=100 width=10)
+ File Output Operator [FS_259]
+ Limit [LIM_258] (rows=100 width=10)
Number of rows:100
- Select Operator [SEL_237] (rows=805 width=10)
+ Select Operator [SEL_257] (rows=805 width=10)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
<-Reducer 4 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_236]
- Group By Operator [GBY_235] (rows=805 width=10)
+ SHUFFLE [RS_256]
+ Group By Operator [GBY_255] (rows=805 width=10)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["count(VALUE._col0)","count(VALUE._col1)","count(VALUE._col2)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Map 3 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_234]
+ SHUFFLE [RS_254]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_233] (rows=1610 width=10)
+ Group By Operator [GBY_253] (rows=1610 width=10)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["count(_col9)","count(_col16)","count(_col3)"],keys:_col8, _col15, _col2
- Top N Key Operator [TNK_232] (rows=1610 width=10)
+ Top N Key Operator [TNK_252] (rows=1610 width=10)
keys:_col8, _col15, _col2,top n:100
- Map Join Operator [MAPJOIN_231] (rows=1610 width=10)
- Conds:RS_207._col1, _col3=SEL_230._col11, _col13(Inner),Output:["_col2","_col3","_col8","_col9","_col15","_col16"]
+ Map Join Operator [MAPJOIN_251] (rows=1610 width=10)
+ Conds:RS_227._col1, _col3=SEL_250._col11, _col13(Inner),Output:["_col2","_col3","_col8","_col9","_col15","_col16"]
<-Map 2 [BROADCAST_EDGE] vectorized, llap
- BROADCAST [RS_207]
+ BROADCAST [RS_227]
PartitionCols:_col1, _col3
- Map Join Operator [MAPJOIN_206] (rows=550 width=10)
- Conds:RS_203._col0=SEL_205._col0(Inner),Output:["_col1","_col2","_col3"]
+ Map Join Operator [MAPJOIN_226] (rows=550 width=10)
+ Conds:RS_223._col0=SEL_225._col0(Inner),Output:["_col1","_col2","_col3"]
<-Map 1 [BROADCAST_EDGE] vectorized, llap
- BROADCAST [RS_203]
+ BROADCAST [RS_223]
PartitionCols:_col0
- Select Operator [SEL_202] (rows=170 width=34)
+ Select Operator [SEL_222] (rows=170 width=34)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_201] (rows=170 width=34)
+ Filter Operator [FIL_221] (rows=170 width=34)
predicate:(v2 is not null and v3 is not null and k1 is not null)
TableScan [TS_0] (rows=170 width=34)
default@cs,cs,Tbl:COMPLETE,Col:NONE,Output:["k1","v2","k3","v3"]
- <-Select Operator [SEL_205] (rows=500 width=10)
+ <-Select Operator [SEL_225] (rows=500 width=10)
Output:["_col0"]
- Filter Operator [FIL_204] (rows=500 width=10)
+ Filter Operator [FIL_224] (rows=500 width=10)
predicate:((value) IN ('2000Q1', '2000Q2', '2000Q3') and key is not null)
TableScan [TS_3] (rows=500 width=10)
default@src,d3,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
- <-Select Operator [SEL_230] (rows=1464 width=10)
+ <-Select Operator [SEL_250] (rows=1464 width=10)
Output:["_col3","_col4","_col10","_col11","_col13"]
- Map Join Operator [MAPJOIN_229] (rows=1464 width=10)
- Conds:MAPJOIN_228._col3, _col5=RS_223._col2, _col4(Inner),Output:["_col1","_col2","_col10","_col11","_col13"]
+ Map Join Operator [MAPJOIN_249] (rows=1464 width=10)
+ Conds:MAPJOIN_248._col3, _col5=RS_243._col2, _col4(Inner),Output:["_col1","_col2","_col10","_col11","_col13"]
<-Map 9 [BROADCAST_EDGE] vectorized, llap
- BROADCAST [RS_223]
+ BROADCAST [RS_243]
PartitionCols:_col2, _col4
- Map Join Operator [MAPJOIN_222] (rows=550 width=10)
- Conds:SEL_221._col0=RS_219._col0(Inner),Output:["_col2","_col3","_col4","_col5"]
+ Map Join Operator [MAPJOIN_242] (rows=550 width=10)
+ Conds:SEL_241._col0=RS_239._col0(Inner),Output:["_col2","_col3","_col4","_col5"]
<-Map 10 [BROADCAST_EDGE] vectorized, llap
- BROADCAST [RS_219]
+ BROADCAST [RS_239]
PartitionCols:_col0
- Select Operator [SEL_218] (rows=42 width=34)
+ Select Operator [SEL_238] (rows=42 width=34)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_217] (rows=42 width=34)
+ Filter Operator [FIL_237] (rows=42 width=34)
predicate:((v1 = 'srv1') and k2 is not null and k3 is not null and v2 is not null and v3 is not null and k1 is not null)
TableScan [TS_21] (rows=85 width=34)
default@sr,sr,Tbl:COMPLETE,Col:NONE,Output:["k1","v1","k2","v2","k3","v3"]
- <-Select Operator [SEL_221] (rows=500 width=10)
+ <-Select Operator [SEL_241] (rows=500 width=10)
Output:["_col0"]
- Filter Operator [FIL_220] (rows=500 width=10)
+ Filter Operator [FIL_240] (rows=500 width=10)
predicate:((value) IN ('2000Q1', '2000Q2', '2000Q3') and key is not null)
TableScan [TS_18] (rows=500 width=10)
default@src,d2,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
- <-Map Join Operator [MAPJOIN_228] (rows=1331 width=10)
- Conds:MAPJOIN_227._col2=RS_216._col0(Inner),Output:["_col1","_col2","_col3","_col5"]
+ <-Map Join Operator [MAPJOIN_248] (rows=1331 width=10)
+ Conds:MAPJOIN_247._col2=RS_236._col0(Inner),Output:["_col1","_col2","_col3","_col5"]
<-Map 8 [BROADCAST_EDGE] vectorized, llap
- BROADCAST [RS_216]
+ BROADCAST [RS_236]
PartitionCols:_col0
- Select Operator [SEL_215] (rows=12 width=7)
+ Select Operator [SEL_235] (rows=12 width=7)
Output:["_col0"]
- Filter Operator [FIL_214] (rows=12 width=7)
+ Filter Operator [FIL_234] (rows=12 width=7)
predicate:((key = 'src1key') and value is not null)
TableScan [TS_15] (rows=25 width=7)
default@src1,src1,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
- <-Map Join Operator [MAPJOIN_227] (rows=1210 width=10)
- Conds:MAPJOIN_226._col1=RS_213._col0(Inner),Output:["_col1","_col2","_col3","_col5"]
+ <-Map Join Operator [MAPJOIN_247] (rows=1210 width=10)
+ Conds:MAPJOIN_246._col1=RS_233._col0(Inner),Output:["_col1","_col2","_col3","_col5"]
<-Map 7 [BROADCAST_EDGE] vectorized, llap
- BROADCAST [RS_213]
+ BROADCAST [RS_233]
PartitionCols:_col0
- Select Operator [SEL_212] (rows=250 width=10)
+ Select Operator [SEL_232] (rows=250 width=10)
Output:["_col0"]
- Filter Operator [FIL_211] (rows=250 width=10)
+ Filter Operator [FIL_231] (rows=250 width=10)
predicate:((value = 'd1value') and key is not null)
TableScan [TS_12] (rows=500 width=10)
default@src,d1,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
- <-Map Join Operator [MAPJOIN_226] (rows=1100 width=10)
- Conds:SEL_225._col0=RS_210._col3(Inner),Output:["_col1","_col2","_col3","_col5"]
+ <-Map Join Operator [MAPJOIN_246] (rows=1100 width=10)
+ Conds:SEL_245._col0=RS_230._col3(Inner),Output:["_col1","_col2","_col3","_col5"]
<-Map 6 [BROADCAST_EDGE] vectorized, llap
- BROADCAST [RS_210]
+ BROADCAST [RS_230]
PartitionCols:_col3
- Select Operator [SEL_209] (rows=42 width=34)
+ Select Operator [SEL_229] (rows=42 width=34)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_208] (rows=42 width=34)
+ Filter Operator [FIL_228] (rows=42 width=34)
predicate:((v3 = 'ssv3') and k2 is not null and k3 is not null and k1 is not null and v1 is not null and v2 is not null)
TableScan [TS_9] (rows=85 width=34)
default@ss_n1,ss_n1,Tbl:COMPLETE,Col:NONE,Output:["k1","v1","k2","v2","k3","v3"]
- <-Select Operator [SEL_225] (rows=1000 width=10)
+ <-Select Operator [SEL_245] (rows=1000 width=10)
Output:["_col0"]
- Filter Operator [FIL_224] (rows=1000 width=10)
+ Filter Operator [FIL_244] (rows=1000 width=10)
predicate:((key = 'srcpartkey') and value is not null)
TableScan [TS_6] (rows=2000 width=10)
default@srcpart,srcpart,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
diff --git a/ql/src/test/results/clientpositive/llap/join_reordering_no_stats.q.out b/ql/src/test/results/clientpositive/llap/join_reordering_no_stats.q.out
index 00a86a2..9d144f7 100644
--- a/ql/src/test/results/clientpositive/llap/join_reordering_no_stats.q.out
+++ b/ql/src/test/results/clientpositive/llap/join_reordering_no_stats.q.out
@@ -228,7 +228,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[47][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[52][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain select count(1) from part_nostats,supplier_nostats,lineitem_nostats where p_partkey = l_partkey and s_suppkey = l_suppkey
PREHOOK: type: QUERY
PREHOOK: Input: default@lineitem_nostats
@@ -602,7 +602,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[47][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[52][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain select count(1) from Employee_Part_n1,supplier_nostats,lineitem_nostats where employeeID= l_partkey and s_suppkey = l_suppkey
PREHOOK: type: QUERY
PREHOOK: Input: default@employee_part_n1
diff --git a/ql/src/test/results/clientpositive/llap/ppd_join5.q.out b/ql/src/test/results/clientpositive/llap/ppd_join5.q.out
index c27f23e..6cb0ad8 100644
--- a/ql/src/test/results/clientpositive/llap/ppd_join5.q.out
+++ b/ql/src/test/results/clientpositive/llap/ppd_join5.q.out
@@ -32,7 +32,7 @@ POSTHOOK: Lineage: t1_n79.id1 SIMPLE []
POSTHOOK: Lineage: t1_n79.id2 SIMPLE []
POSTHOOK: Lineage: t2_n48.d SIMPLE []
POSTHOOK: Lineage: t2_n48.id SIMPLE []
-Warning: Shuffle Join MERGEJOIN[38][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[43][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain
select a.*,b.d d1,c.d d2 from
t1_n79 a join t2_n48 b on (a.id1 = b.id)
@@ -163,7 +163,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[41][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[46][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain
select * from (
select a.*,b.d d1,c.d d2 from
@@ -300,7 +300,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[41][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[46][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select * from (
select a.*,b.d d1,c.d d2 from
t1_n79 a join t2_n48 b on (a.id1 = b.id)
diff --git a/ql/src/test/results/clientpositive/llap/semijoin.q.out b/ql/src/test/results/clientpositive/llap/semijoin.q.out
index 998e975..71762a9 100644
--- a/ql/src/test/results/clientpositive/llap/semijoin.q.out
+++ b/ql/src/test/results/clientpositive/llap/semijoin.q.out
@@ -3323,7 +3323,7 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@t2_n33
POSTHOOK: Input: default@t3_n12
#### A masked pattern was here ####
-Warning: Shuffle Join MERGEJOIN[52][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[57][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select key, value from src outr left semi join
(select a.key, b.value from src a join (select distinct value from src) b on a.value > b.value group by a.key, b.value) inr
on outr.key=inr.key and outr.value=inr.value
@@ -3457,7 +3457,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[52][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[57][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select key, value from src outr left semi join
(select a.key, b.value from src a join (select distinct value from src) b on a.value > b.value group by a.key, b.value) inr
on outr.key=inr.key and outr.value=inr.value
diff --git a/ql/src/test/results/clientpositive/llap/semijoin4.q.out b/ql/src/test/results/clientpositive/llap/semijoin4.q.out
index e13ade3..d5c36b3 100644
--- a/ql/src/test/results/clientpositive/llap/semijoin4.q.out
+++ b/ql/src/test/results/clientpositive/llap/semijoin4.q.out
@@ -18,7 +18,7 @@ STORED AS orc
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@table_18
-Warning: Shuffle Join MERGEJOIN[79][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[99][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain
SELECT
COALESCE(498,
diff --git a/ql/src/test/results/clientpositive/llap/subquery_in.q.out b/ql/src/test/results/clientpositive/llap/subquery_in.q.out
index 197ffed..addf439 100644
--- a/ql/src/test/results/clientpositive/llap/subquery_in.q.out
+++ b/ql/src/test/results/clientpositive/llap/subquery_in.q.out
@@ -4627,7 +4627,7 @@ POSTHOOK: Input: default@part
85768 almond antique chartreuse lavender yellow Manufacturer#1 Brand#12 LARGE BRUSHED STEEL 34 SM BAG 1753.76 refull
86428 almond aquamarine burnished black steel Manufacturer#1 Brand#12 STANDARD ANODIZED STEEL 28 WRAP BAG 1414.42 arefully
90681 almond antique chartreuse khaki white Manufacturer#3 Brand#31 MEDIUM BURNISHED TIN 17 SM CASE 1671.68 are slyly after the sl
-Warning: Shuffle Join MERGEJOIN[53][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[58][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select * from part where p_size in (select min(pp.p_size) from part pp where pp.p_partkey > part.p_partkey)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
@@ -4792,7 +4792,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[53][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[58][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select * from part where p_size in (select min(pp.p_size) from part pp where pp.p_partkey > part.p_partkey)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
@@ -5688,7 +5688,7 @@ POSTHOOK: query: drop table tt_n2
POSTHOOK: type: DROPTABLE
POSTHOOK: Input: default@tt_n2
POSTHOOK: Output: default@tt_n2
-Warning: Shuffle Join MERGEJOIN[49][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[54][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select * from part where p_size IN (select max(p_size) from part p where p.p_type <> part.p_name)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
@@ -5849,7 +5849,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[49][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[54][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select * from part where p_size IN (select max(p_size) from part p where p.p_type <> part.p_name)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
@@ -5859,7 +5859,7 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@part
#### A masked pattern was here ####
15103 almond aquamarine dodger light gainsboro Manufacturer#5 Brand#53 ECONOMY BURNISHED STEEL 46 LG PACK 1018.1 packages hinder carefu
-Warning: Shuffle Join MERGEJOIN[71][tables = [$hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
+Warning: Shuffle Join MERGEJOIN[76][tables = [$hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
PREHOOK: query: explain select * from part where p_size IN (select pp.p_size from part p join part pp on pp.p_type = p.p_type where part.p_type <> p.p_name)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
@@ -6038,7 +6038,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[71][tables = [$hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
+Warning: Shuffle Join MERGEJOIN[76][tables = [$hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
PREHOOK: query: select * from part where p_size IN (select pp.p_size from part p join part pp on pp.p_type = p.p_type where part.p_type <> p.p_name)
PREHOOK: type: QUERY
PREHOOK: Input: default@part
diff --git a/ql/src/test/results/clientpositive/llap/subquery_multi.q.out b/ql/src/test/results/clientpositive/llap/subquery_multi.q.out
index be03281..729be01 100644
--- a/ql/src/test/results/clientpositive/llap/subquery_multi.q.out
+++ b/ql/src/test/results/clientpositive/llap/subquery_multi.q.out
@@ -3837,7 +3837,7 @@ POSTHOOK: Input: default@src
96 val_96 1
97 val_97 2
98 val_98 2
-Warning: Shuffle Join MERGEJOIN[50][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[55][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select * from part where p_name IN (select p_name from part p where part.p_type <> '1')
PREHOOK: type: QUERY
PREHOOK: Input: default@part
@@ -3970,7 +3970,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[50][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[55][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select * from part where p_name IN (select p_name from part p where part.p_type <> '1')
PREHOOK: type: QUERY
PREHOOK: Input: default@part
diff --git a/ql/src/test/results/clientpositive/llap/subquery_multiinsert.q.out b/ql/src/test/results/clientpositive/llap/subquery_multiinsert.q.out
index 6367011..9d8261c 100644
--- a/ql/src/test/results/clientpositive/llap/subquery_multiinsert.q.out
+++ b/ql/src/test/results/clientpositive/llap/subquery_multiinsert.q.out
@@ -28,7 +28,7 @@ POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@src_5
RUN: Stage-0:DDL
-Warning: Shuffle Join MERGEJOIN[78][tables = [b, sq_2_notin_nullcheck]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[83][tables = [b, sq_2_notin_nullcheck]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain
from src b
INSERT OVERWRITE TABLE src_4
@@ -354,7 +354,7 @@ STAGE PLANS:
Column Types: string, string
Table: default.src_4
-Warning: Shuffle Join MERGEJOIN[78][tables = [b, sq_2_notin_nullcheck]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[83][tables = [b, sq_2_notin_nullcheck]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: from src b
INSERT OVERWRITE TABLE src_4
select *
@@ -543,7 +543,7 @@ POSTHOOK: Input: default@src_5
199 val_199
199 val_199
2 val_2
-Warning: Map Join MAPJOIN[78][bigTable=b] in task 'Map 1' is a cross product
+Warning: Map Join MAPJOIN[83][bigTable=b] in task 'Map 1' is a cross product
PREHOOK: query: explain
from src b
INSERT OVERWRITE TABLE src_4
@@ -854,7 +854,7 @@ STAGE PLANS:
Column Types: string, string
Table: default.src_5
-Warning: Map Join MAPJOIN[78][bigTable=b] in task 'Map 1' is a cross product
+Warning: Map Join MAPJOIN[83][bigTable=b] in task 'Map 1' is a cross product
PREHOOK: query: from src b
INSERT OVERWRITE TABLE src_4
select *
diff --git a/ql/src/test/results/clientpositive/llap/subquery_scalar.q.out b/ql/src/test/results/clientpositive/llap/subquery_scalar.q.out
index 4ef2b56..e8531f2 100644
--- a/ql/src/test/results/clientpositive/llap/subquery_scalar.q.out
+++ b/ql/src/test/results/clientpositive/llap/subquery_scalar.q.out
@@ -2123,7 +2123,7 @@ POSTHOOK: Input: default@part_null_n0
85768 almond antique chartreuse lavender yellow Manufacturer#1 Brand#12 LARGE BRUSHED STEEL 34 SM BAG 1753.76 refull
86428 almond aquamarine burnished black steel Manufacturer#1 Brand#12 STANDARD ANODIZED STEEL 28 WRAP BAG 1414.42 arefully
90681 almond antique chartreuse khaki white Manufacturer#3 Brand#31 MEDIUM BURNISHED TIN 17 SM CASE 1671.68 are slyly after the sl
-Warning: Shuffle Join MERGEJOIN[47][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[57][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain select * from part where p_brand <> (select min(p_brand) from part ) AND p_size IN (select (p_size) from part p where p.p_type = part.p_type ) AND p_size <> 340
PREHOOK: type: QUERY
PREHOOK: Input: default@part
@@ -2256,7 +2256,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[47][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[57][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: select * from part where p_brand <> (select min(p_brand) from part ) AND p_size IN (select (p_size) from part p where p.p_type = part.p_type ) AND p_size <> 340
PREHOOK: type: QUERY
PREHOOK: Input: default@part
diff --git a/ql/src/test/results/clientpositive/llap/subquery_select.q.out b/ql/src/test/results/clientpositive/llap/subquery_select.q.out
index 433b213..5539cfa 100644
--- a/ql/src/test/results/clientpositive/llap/subquery_select.q.out
+++ b/ql/src/test/results/clientpositive/llap/subquery_select.q.out
@@ -1853,7 +1853,7 @@ POSTHOOK: Input: default@part
6 46
6 46
7 46
-Warning: Shuffle Join MERGEJOIN[46][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[51][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain
select *
from src b
@@ -2001,7 +2001,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[46][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[51][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select *
from src b
where b.key in
@@ -4745,7 +4745,7 @@ POSTHOOK: Input: default@part
46 1 19.692307692307693 512
46 1 19.692307692307693 512
46 1 19.692307692307693 512
-Warning: Shuffle Join MERGEJOIN[46][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[56][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain select t1.p_size,
(select count(*) from part p, part pp where p.p_size = pp.p_size and p.p_type = pp.p_type)
from part t1
@@ -4881,7 +4881,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[46][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[56][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: select t1.p_size,
(select count(*) from part p, part pp where p.p_size = pp.p_size and p.p_type = pp.p_type)
from part t1
@@ -4920,7 +4920,7 @@ POSTHOOK: Input: default@part
6 28
6 28
7 28
-Warning: Shuffle Join MERGEJOIN[102][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[112][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain select t1.p_size,
(select count(*) from part p, part pp where p.p_size = pp.p_size and p.p_type = pp.p_type
and (select sum(p_size) from part a1 where a1.p_partkey = p.p_partkey
@@ -5177,7 +5177,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[102][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[112][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: select t1.p_size,
(select count(*) from part p, part pp where p.p_size = pp.p_size and p.p_type = pp.p_type
and (select sum(p_size) from part a1 where a1.p_partkey = p.p_partkey
@@ -5220,7 +5220,7 @@ POSTHOOK: Input: default@part
6 28
6 28
7 28
-Warning: Shuffle Join MERGEJOIN[144][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
+Warning: Shuffle Join MERGEJOIN[154][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
PREHOOK: query: explain select t1.p_size,
(select count(*) from part t2 where t2.p_partkey = t1.p_partkey group by t2.p_partkey),
(select count(*) from part p, part pp where p.p_size = pp.p_size and p.p_type = pp.p_type
@@ -5596,7 +5596,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[144][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
+Warning: Shuffle Join MERGEJOIN[154][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
PREHOOK: query: select t1.p_size,
(select count(*) from part t2 where t2.p_partkey = t1.p_partkey group by t2.p_partkey),
(select count(*) from part p, part pp where p.p_size = pp.p_size and p.p_type = pp.p_type
diff --git a/ql/src/test/results/clientpositive/llap/tez_fixed_bucket_pruning.q.out b/ql/src/test/results/clientpositive/llap/tez_fixed_bucket_pruning.q.out
index f2f2c31..e8cf3da 100644
--- a/ql/src/test/results/clientpositive/llap/tez_fixed_bucket_pruning.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_fixed_bucket_pruning.q.out
@@ -424,7 +424,7 @@ POSTHOOK: type: ANALYZE_TABLE
POSTHOOK: Input: default@l3_monthly_dw_dimplan
POSTHOOK: Output: default@l3_monthly_dw_dimplan
#### A masked pattern was here ####
-Warning: Map Join MAPJOIN[49][bigTable=?] in task 'Map 1' is a cross product
+Warning: Map Join MAPJOIN[54][bigTable=?] in task 'Map 1' is a cross product
PREHOOK: query: EXPLAIN EXTENDED
SELECT DW.PROJECT_OBJECT_ID, S1.PLAN_KEY as PLAN_KEY, S2.PROJECT_KEY AS PROJECT_KEY
FROM l3_clarity__L3_SNAP_NUMBER_2018022300104 snap inner join
@@ -842,7 +842,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Map Join MAPJOIN[49][bigTable=?] in task 'Map 1' is a cross product
+Warning: Map Join MAPJOIN[54][bigTable=?] in task 'Map 1' is a cross product
PREHOOK: query: SELECT DW.PROJECT_OBJECT_ID, S1.PLAN_KEY as PLAN_KEY, S2.PROJECT_KEY AS PROJECT_KEY
FROM l3_clarity__L3_SNAP_NUMBER_2018022300104 snap inner join
l3_clarity__L3_MONTHLY_DW_FACTPLAN_DW_STG_2018022300104_1 DW on 1=1
@@ -884,7 +884,7 @@ POSTHOOK: Input: default@l3_monthly_dw_dimplan
7147200 195775 27114
7147200 234349 27114
7147200 350519 27114
-Warning: Map Join MAPJOIN[49][bigTable=?] in task 'Map 1' is a cross product
+Warning: Map Join MAPJOIN[54][bigTable=?] in task 'Map 1' is a cross product
PREHOOK: query: EXPLAIN EXTENDED
SELECT DW.PROJECT_OBJECT_ID, S1.PLAN_KEY as PLAN_KEY, S2.PROJECT_KEY AS PROJECT_KEY
FROM l3_clarity__L3_SNAP_NUMBER_2018022300104 snap inner join
@@ -1302,7 +1302,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Map Join MAPJOIN[49][bigTable=?] in task 'Map 1' is a cross product
+Warning: Map Join MAPJOIN[54][bigTable=?] in task 'Map 1' is a cross product
PREHOOK: query: SELECT DW.PROJECT_OBJECT_ID, S1.PLAN_KEY as PLAN_KEY, S2.PROJECT_KEY AS PROJECT_KEY
FROM l3_clarity__L3_SNAP_NUMBER_2018022300104 snap inner join
l3_clarity__L3_MONTHLY_DW_FACTPLAN_DW_STG_2018022300104_1 DW on 1=1
diff --git a/ql/src/test/results/clientpositive/llap/vectorized_dynamic_semijoin_reduction.q.out b/ql/src/test/results/clientpositive/llap/vectorized_dynamic_semijoin_reduction.q.out
index 8413d53..e93471d 100644
--- a/ql/src/test/results/clientpositive/llap/vectorized_dynamic_semijoin_reduction.q.out
+++ b/ql/src/test/results/clientpositive/llap/vectorized_dynamic_semijoin_reduction.q.out
@@ -1088,120 +1088,71 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- filterExpr: (key_str is not null and key_int is not null and key_str BETWEEN DynamicValue(RS_7_b_key_str_min) AND DynamicValue(RS_7_b_key_str_max) and in_bloom_filter(key_str, DynamicValue(RS_7_b_key_str_bloom_filter))) (type: boolean)
+ filterExpr: (key_str is not null and key_int is not null and key_str BETWEEN DynamicValue(RS[43]_col0) AND DynamicValue(RS[43]_col1) and key_int BETWEEN DynamicValue(RS[43]_col2) AND DynamicValue(RS[43]_col3) and in_bloom_filter(hash(key_str,key_int), DynamicValue(RS[43]_col4))) (type: boolean)
Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
- TableScan Vectorization:
- native: true
Filter Operator
- Filter Vectorization:
- className: VectorFilterOperator
- native: true
- predicateExpression: FilterExprAndExpr(children: SelectColumnIsNotNull(col 0:string), SelectColumnIsNotNull(col 1:int), FilterStringColumnBetweenDynamicValue(col 0:string, left NULL, right NULL), VectorInBloomFilterColDynamicValue)
- predicate: (key_str is not null and key_int is not null and key_str BETWEEN DynamicValue(RS_7_b_key_str_min) AND DynamicValue(RS_7_b_key_str_max) and in_bloom_filter(key_str, DynamicValue(RS_7_b_key_str_bloom_filter))) (type: boolean)
+ predicate: (key_str is not null and key_int is not null and key_str BETWEEN DynamicValue(RS[43]_col0) AND DynamicValue(RS[43]_col1) and key_int BETWEEN DynamicValue(RS[43]_col2) AND DynamicValue(RS[43]_col3) and in_bloom_filter(hash(key_str,key_int), DynamicValue(RS[43]_col4))) (type: boolean)
Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key_str (type: string), key_int (type: int)
outputColumnNames: _col0, _col1
- Select Vectorization:
- className: VectorSelectOperator
- native: true
- projectedOutputColumnNums: [0, 1]
Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: int)
- Reduce Sink Vectorization:
- className: VectorReduceSinkMultiKeyOperator
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: vectorized, llap
+ Execution mode: llap
LLAP IO: all inputs
Map Vectorization:
enabled: true
enabledConditionsMet: hive.vectorized.use.vectorized.input.format IS true
- inputFormatFeatureSupport: [DECIMAL_64]
- featureSupportInUse: [DECIMAL_64]
inputFileFormats: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
- allNative: true
- usesVectorUDFAdaptor: false
- vectorized: true
+ notVectorizedReason: Predicate expression for FILTER operator: UDF GenericUDFMurmurHash(Column[key_str], Column[key_int]) not supported
+ vectorized: false
Map 4
Map Operator Tree:
TableScan
alias: b
filterExpr: (key_str is not null and key_int is not null) (type: boolean)
Statistics: Num rows: 57 Data size: 5130 Basic stats: COMPLETE Column stats: COMPLETE
- TableScan Vectorization:
- native: true
Filter Operator
- Filter Vectorization:
- className: VectorFilterOperator
- native: true
- predicateExpression: FilterExprAndExpr(children: SelectColumnIsNotNull(col 0:string), SelectColumnIsNotNull(col 1:int))
predicate: (key_str is not null and key_int is not null) (type: boolean)
Statistics: Num rows: 57 Data size: 5130 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key_str (type: string), key_int (type: int)
outputColumnNames: _col0, _col1
- Select Vectorization:
- className: VectorSelectOperator
- native: true
- projectedOutputColumnNums: [0, 1]
Statistics: Num rows: 57 Data size: 5130 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: int)
null sort order: zz
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: int)
- Reduce Sink Vectorization:
- className: VectorReduceSinkMultiKeyOperator
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
Statistics: Num rows: 57 Data size: 5130 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: _col0 (type: string)
- outputColumnNames: _col0
- Select Vectorization:
- className: VectorSelectOperator
- native: true
- projectedOutputColumnNums: [0]
- Statistics: Num rows: 57 Data size: 4902 Basic stats: COMPLETE Column stats: COMPLETE
+ expressions: _col0 (type: string), _col1 (type: int), hash(_col0,_col1) (type: int)
+ outputColumnNames: _col0, _col1, _col3
+ Statistics: Num rows: 57 Data size: 5358 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
- aggregations: min(_col0), max(_col0), bloom_filter(_col0, expectedEntries=57)
- Group By Vectorization:
- aggregators: VectorUDAFMinString(col 0:string) -> string, VectorUDAFMaxString(col 0:string) -> string, VectorUDAFBloomFilter(col 0:string) -> binary
- className: VectorGroupByOperator
- groupByMode: HASH
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: [0, 1, 2]
+ aggregations: min(_col0), max(_col0), min(_col1), max(_col1), bloom_filter(_col3, expectedEntries=57)
minReductionHashAggr: 0.98245615
mode: hash
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4
+ Statistics: Num rows: 1 Data size: 520 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
- Reduce Sink Vectorization:
- className: VectorReduceSinkEmptyKeyOperator
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
- Statistics: Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: binary)
- Execution mode: vectorized, llap
+ Statistics: Num rows: 1 Data size: 520 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: int), _col3 (type: int), _col4 (type: binary)
+ Execution mode: llap
LLAP IO: all inputs
Map Vectorization:
enabled: true
enabledConditionsMet: hive.vectorized.use.vectorized.input.format IS true
- inputFormatFeatureSupport: [DECIMAL_64]
- featureSupportInUse: [DECIMAL_64]
inputFileFormats: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
- allNative: false
- usesVectorUDFAdaptor: false
- vectorized: true
+ notVectorizedReason: Select expression for SELECT operator: UDF GenericUDFMurmurHash(Column[_col0], Column[_col1]) not supported
+ vectorized: false
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -1267,17 +1218,17 @@ STAGE PLANS:
vectorized: true
Reduce Operator Tree:
Group By Operator
- aggregations: min(VALUE._col0), max(VALUE._col1), bloom_filter(VALUE._col2, expectedEntries=57)
+ aggregations: min(VALUE._col0), max(VALUE._col1), min(VALUE._col2), max(VALUE._col3), bloom_filter(VALUE._col4, expectedEntries=57)
Group By Vectorization:
- aggregators: VectorUDAFMinString(col 0:string) -> string, VectorUDAFMaxString(col 1:string) -> string, VectorUDAFBloomFilterMerge(col 2:binary) -> binary
+ aggregators: VectorUDAFMinString(col 0:string) -> string, VectorUDAFMaxString(col 1:string) -> string, VectorUDAFMinLong(col 2:int) -> int, VectorUDAFMaxLong(col 3:int) -> int, VectorUDAFBloomFilterMerge(col 4:binary) -> binary
className: VectorGroupByOperator
groupByMode: FINAL
native: false
vectorProcessingMode: STREAMING
- projectedOutputColumnNums: [0, 1, 2]
+ projectedOutputColumnNums: [0, 1, 2, 3, 4]
mode: final
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4
+ Statistics: Num rows: 1 Data size: 520 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
null sort order:
sort order:
@@ -1285,8 +1236,8 @@ STAGE PLANS:
className: VectorReduceSinkEmptyKeyOperator
native: true
nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
- Statistics: Num rows: 1 Data size: 552 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: binary)
+ Statistics: Num rows: 1 Data size: 520 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: int), _col3 (type: int), _col4 (type: binary)
Stage: Stage-0
Fetch Operator
diff --git a/ql/src/test/results/clientpositive/llap/vectorized_mapjoin3.q.out b/ql/src/test/results/clientpositive/llap/vectorized_mapjoin3.q.out
index fdf311b..4f2f0e4 100644
--- a/ql/src/test/results/clientpositive/llap/vectorized_mapjoin3.q.out
+++ b/ql/src/test/results/clientpositive/llap/vectorized_mapjoin3.q.out
@@ -111,7 +111,7 @@ POSTHOOK: Input: default@table_19
POSTHOOK: Input: default@table_6
#### A masked pattern was here ####
418.9
-Warning: Shuffle Join MERGEJOIN[49][tables = [$hdt$_0]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[59][tables = [$hdt$_0]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: SELECT
t1.t_id,
null
@@ -800,7 +800,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[46][tables = [$hdt$_0]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[56][tables = [$hdt$_0]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: EXPLAIN VECTORIZATION DETAIL
SELECT
t1.t_id,
diff --git a/ql/src/test/results/clientpositive/perf/tez/cbo_query14.q.out b/ql/src/test/results/clientpositive/perf/tez/cbo_query14.q.out
index adb4a9b..a8b554e 100644
--- a/ql/src/test/results/clientpositive/perf/tez/cbo_query14.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/cbo_query14.q.out
@@ -1,6 +1,6 @@
-Warning: Shuffle Join MERGEJOIN[1178][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 6' is a cross product
-Warning: Shuffle Join MERGEJOIN[1185][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 16' is a cross product
-Warning: Shuffle Join MERGEJOIN[1192][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 22' is a cross product
+Warning: Shuffle Join MERGEJOIN[1238][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 6' is a cross product
+Warning: Shuffle Join MERGEJOIN[1245][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 16' is a cross product
+Warning: Shuffle Join MERGEJOIN[1252][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 22' is a cross product
PREHOOK: query: explain cbo
with cross_items as
(select i_item_sk ss_item_sk
diff --git a/ql/src/test/results/clientpositive/perf/tez/cbo_query24.q.out b/ql/src/test/results/clientpositive/perf/tez/cbo_query24.q.out
index 3092bf4..440e72d 100644
--- a/ql/src/test/results/clientpositive/perf/tez/cbo_query24.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/cbo_query24.q.out
@@ -1,4 +1,4 @@
-Warning: Shuffle Join MERGEJOIN[305][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 6' is a cross product
+Warning: Shuffle Join MERGEJOIN[335][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 6' is a cross product
PREHOOK: query: explain cbo
with ssales as
(select c_last_name
diff --git a/ql/src/test/results/clientpositive/perf/tez/cbo_query54.q.out b/ql/src/test/results/clientpositive/perf/tez/cbo_query54.q.out
index bb2cacc..fd90e5f 100644
--- a/ql/src/test/results/clientpositive/perf/tez/cbo_query54.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/cbo_query54.q.out
@@ -1,7 +1,7 @@
-Warning: Shuffle Join MERGEJOIN[273][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
-Warning: Shuffle Join MERGEJOIN[274][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4]] in Stage 'Reducer 5' is a cross product
-Warning: Shuffle Join MERGEJOIN[275][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5]] in Stage 'Reducer 6' is a cross product
-Warning: Shuffle Join MERGEJOIN[276][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6]] in Stage 'Reducer 7' is a cross product
+Warning: Shuffle Join MERGEJOIN[283][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
+Warning: Shuffle Join MERGEJOIN[284][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4]] in Stage 'Reducer 5' is a cross product
+Warning: Shuffle Join MERGEJOIN[285][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5]] in Stage 'Reducer 6' is a cross product
+Warning: Shuffle Join MERGEJOIN[286][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3, $hdt$_4, $hdt$_5, $hdt$_6]] in Stage 'Reducer 7' is a cross product
PREHOOK: query: explain cbo
with my_customers as (
select distinct c_customer_sk
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query14.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query14.q.out
index f2eb090..cf80ce3 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query14.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query14.q.out
@@ -1,6 +1,6 @@
-Warning: Shuffle Join MERGEJOIN[1172][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 7' is a cross product
-Warning: Shuffle Join MERGEJOIN[1179][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 18' is a cross product
-Warning: Shuffle Join MERGEJOIN[1186][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 25' is a cross product
+Warning: Shuffle Join MERGEJOIN[1232][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 7' is a cross product
+Warning: Shuffle Join MERGEJOIN[1239][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 18' is a cross product
+Warning: Shuffle Join MERGEJOIN[1246][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 25' is a cross product
PREHOOK: query: explain cbo
with cross_items as
(select i_item_sk ss_item_sk
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query24.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query24.q.out
index 7af039c..0fd5160 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query24.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query24.q.out
@@ -1,4 +1,4 @@
-Warning: Shuffle Join MERGEJOIN[301][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 6' is a cross product
+Warning: Shuffle Join MERGEJOIN[331][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 6' is a cross product
PREHOOK: query: explain cbo
with ssales as
(select c_last_name
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query54.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query54.q.out
index ce41e9e..b1bfed3 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query54.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query54.q.out
@@ -1,7 +1,7 @@
-Warning: Shuffle Join MERGEJOIN[279][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
-Warning: Shuffle Join MERGEJOIN[285][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
-Warning: Shuffle Join MERGEJOIN[284][tables = [$hdt$_4, $hdt$_5]] in Stage 'Reducer 15' is a cross product
-Warning: Shuffle Join MERGEJOIN[287][tables = [$hdt$_4, $hdt$_5, $hdt$_6]] in Stage 'Reducer 16' is a cross product
+Warning: Shuffle Join MERGEJOIN[289][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[295][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[294][tables = [$hdt$_4, $hdt$_5]] in Stage 'Reducer 15' is a cross product
+Warning: Shuffle Join MERGEJOIN[297][tables = [$hdt$_4, $hdt$_5, $hdt$_6]] in Stage 'Reducer 16' is a cross product
PREHOOK: query: explain cbo
with my_customers as (
select distinct c_customer_sk
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query14.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query14.q.out
index 6b8921e..cf54f58 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query14.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query14.q.out
@@ -1,6 +1,6 @@
-Warning: Shuffle Join MERGEJOIN[1172][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 7' is a cross product
-Warning: Shuffle Join MERGEJOIN[1179][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 18' is a cross product
-Warning: Shuffle Join MERGEJOIN[1186][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 25' is a cross product
+Warning: Shuffle Join MERGEJOIN[1232][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 7' is a cross product
+Warning: Shuffle Join MERGEJOIN[1239][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 18' is a cross product
+Warning: Shuffle Join MERGEJOIN[1246][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 25' is a cross product
PREHOOK: query: explain
with cross_items as
(select i_item_sk ss_item_sk
@@ -295,36 +295,36 @@ Stage-0
limit:100
Stage-1
Reducer 10 vectorized
- File Output Operator [FS_1359]
- Limit [LIM_1358] (rows=100 width=220)
+ File Output Operator [FS_1419]
+ Limit [LIM_1418] (rows=100 width=220)
Number of rows:100
- Select Operator [SEL_1357] (rows=205 width=220)
+ Select Operator [SEL_1417] (rows=205 width=220)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
<-Reducer 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1356]
- Select Operator [SEL_1355] (rows=205 width=220)
+ SHUFFLE [RS_1416]
+ Select Operator [SEL_1415] (rows=205 width=220)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Group By Operator [GBY_1354] (rows=205 width=228)
+ Group By Operator [GBY_1414] (rows=205 width=228)
Output:["_col0","_col1","_col2","_col3","_col5","_col6"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4
<-Union 8 [SIMPLE_EDGE]
<-Reducer 18 [CONTAINS]
- Reduce Output Operator [RS_1185]
+ Reduce Output Operator [RS_1245]
PartitionCols:_col0, _col1, _col2, _col3, _col4
- Group By Operator [GBY_1184] (rows=205 width=228)
+ Group By Operator [GBY_1244] (rows=205 width=228)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["sum(_col4)","sum(_col5)"],keys:_col0, _col1, _col2, _col3, 0L
- Top N Key Operator [TNK_1183] (rows=82 width=220)
+ Top N Key Operator [TNK_1243] (rows=82 width=220)
keys:_col0, _col1, _col2, _col3,top n:100
- Select Operator [SEL_1181] (rows=24 width=222)
+ Select Operator [SEL_1241] (rows=24 width=222)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_1180] (rows=24 width=243)
+ Filter Operator [FIL_1240] (rows=24 width=243)
predicate:(_col3 > _col5)
- Merge Join Operator [MERGEJOIN_1179] (rows=72 width=243)
+ Merge Join Operator [MERGEJOIN_1239] (rows=72 width=243)
Conds:(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
<-Reducer 17 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1371]
- Filter Operator [FIL_1370] (rows=72 width=131)
+ PARTITION_ONLY_SHUFFLE [RS_1431]
+ Filter Operator [FIL_1430] (rows=72 width=131)
predicate:_col3 is not null
- Group By Operator [GBY_1369] (rows=72 width=131)
+ Group By Operator [GBY_1429] (rows=72 width=131)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 16 [SIMPLE_EDGE]
SHUFFLE [RS_236]
@@ -333,350 +333,350 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col3)","count()"],keys:_col0, _col1, _col2
Select Operator [SEL_233] (rows=12217 width=10)
Output:["_col0","_col1","_col2","_col3"]
- Merge Join Operator [MERGEJOIN_1152] (rows=12217 width=10)
- Conds:RS_230._col1=RS_1331._col0(Inner),Output:["_col2","_col3","_col13","_col14","_col15"]
+ Merge Join Operator [MERGEJOIN_1212] (rows=12217 width=10)
+ Conds:RS_230._col1=RS_1391._col0(Inner),Output:["_col2","_col3","_col13","_col14","_col15"]
<-Map 62 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1331]
+ SHUFFLE [RS_1391]
PartitionCols:_col0
- Select Operator [SEL_1322] (rows=462000 width=15)
+ Select Operator [SEL_1382] (rows=462000 width=15)
Output:["_col0","_col1","_col2","_col3"]
TableScan [TS_6] (rows=462000 width=15)
default@item,item,Tbl:COMPLETE,Col:COMPLETE,Output:["i_item_sk","i_brand_id","i_class_id","i_category_id"]
<-Reducer 15 [SIMPLE_EDGE]
SHUFFLE [RS_230]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_1151] (rows=12217 width=4)
- Conds:RS_227._col6, _col7, _col8=RS_1368._col0, _col1, _col2(Inner),Output:["_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_1211] (rows=12217 width=4)
+ Conds:RS_227._col6, _col7, _col8=RS_1428._col0, _col1, _col2(Inner),Output:["_col1","_col2","_col3"]
<-Reducer 14 [SIMPLE_EDGE]
SHUFFLE [RS_227]
PartitionCols:_col6, _col7, _col8
- Merge Join Operator [MERGEJOIN_1144] (rows=7733674 width=110)
- Conds:RS_224._col1=RS_1338._col0(Inner),Output:["_col1","_col2","_col3","_col6","_col7","_col8"]
+ Merge Join Operator [MERGEJOIN_1204] (rows=7733674 width=110)
+ Conds:RS_224._col1=RS_1398._col0(Inner),Output:["_col1","_col2","_col3","_col6","_col7","_col8"]
<-Map 62 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1338]
+ SHUFFLE [RS_1398]
PartitionCols:_col0
- Select Operator [SEL_1327] (rows=458612 width=15)
+ Select Operator [SEL_1387] (rows=458612 width=15)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_1318] (rows=458612 width=15)
+ Filter Operator [FIL_1378] (rows=458612 width=15)
predicate:(i_category_id is not null and i_brand_id is not null and i_class_id is not null)
Please refer to the previous TableScan [TS_6]
<-Reducer 13 [SIMPLE_EDGE]
SHUFFLE [RS_224]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_1143] (rows=7790806 width=98)
- Conds:RS_1364._col0=RS_1289._col0(Inner),Output:["_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_1203] (rows=7790806 width=98)
+ Conds:RS_1424._col0=RS_1349._col0(Inner),Output:["_col1","_col2","_col3"]
<-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1289]
+ SHUFFLE [RS_1349]
PartitionCols:_col0
- Select Operator [SEL_1285] (rows=50 width=4)
+ Select Operator [SEL_1345] (rows=50 width=4)
Output:["_col0"]
- Filter Operator [FIL_1283] (rows=50 width=12)
+ Filter Operator [FIL_1343] (rows=50 width=12)
predicate:((d_year = 2000) and (d_moy = 11))
TableScan [TS_3] (rows=73049 width=12)
default@date_dim,date_dim,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_year","d_moy"]
<-Map 78 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1364]
+ SHUFFLE [RS_1424]
PartitionCols:_col0
- Select Operator [SEL_1363] (rows=286549727 width=123)
+ Select Operator [SEL_1423] (rows=286549727 width=123)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_1362] (rows=286549727 width=123)
+ Filter Operator [FIL_1422] (rows=286549727 width=123)
predicate:(cs_sold_date_sk is not null and cs_sold_date_sk BETWEEN DynamicValue(RS_222_date_dim_d_date_sk_min) AND DynamicValue(RS_222_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_222_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_142] (rows=287989836 width=123)
default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["cs_sold_date_sk","cs_item_sk","cs_quantity","cs_list_price"]
<-Reducer 19 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_1361]
- Group By Operator [GBY_1360] (rows=1 width=12)
+ BROADCAST [RS_1421]
+ Group By Operator [GBY_1420] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 11 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1307]
- Group By Operator [GBY_1302] (rows=1 width=12)
+ SHUFFLE [RS_1367]
+ Group By Operator [GBY_1362] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_1290] (rows=50 width=4)
+ Select Operator [SEL_1350] (rows=50 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_1285]
+ Please refer to the previous Select Operator [SEL_1345]
<-Reducer 35 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1368]
+ SHUFFLE [RS_1428]
PartitionCols:_col0, _col1, _col2
- Select Operator [SEL_1367] (rows=1 width=12)
+ Select Operator [SEL_1427] (rows=1 width=12)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_1366] (rows=1 width=20)
+ Filter Operator [FIL_1426] (rows=1 width=20)
predicate:(_col3 = 3L)
- Group By Operator [GBY_1365] (rows=121728 width=19)
+ Group By Operator [GBY_1425] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Union 34 [SIMPLE_EDGE]
<-Reducer 33 [CONTAINS] vectorized
- Reduce Output Operator [RS_1405]
+ Reduce Output Operator [RS_1465]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_1404] (rows=121728 width=19)
+ Group By Operator [GBY_1464] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(_col3)"],keys:_col0, _col1, _col2
- Group By Operator [GBY_1403] (rows=121728 width=19)
+ Group By Operator [GBY_1463] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 32 [SIMPLE_EDGE]
SHUFFLE [RS_168]
PartitionCols:_col0, _col1, _col2
Group By Operator [GBY_167] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count()"],keys:_col4, _col5, _col6
- Merge Join Operator [MERGEJOIN_1146] (rows=14628613 width=11)
- Conds:RS_163._col1=RS_1339._col0(Inner),Output:["_col4","_col5","_col6"]
+ Merge Join Operator [MERGEJOIN_1206] (rows=14628613 width=11)
+ Conds:RS_163._col1=RS_1399._col0(Inner),Output:["_col4","_col5","_col6"]
<-Map 62 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1339]
+ SHUFFLE [RS_1399]
PartitionCols:_col0
- Select Operator [SEL_1328] (rows=458612 width=15)
+ Select Operator [SEL_1388] (rows=458612 width=15)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_1319] (rows=458612 width=15)
+ Filter Operator [FIL_1379] (rows=458612 width=15)
predicate:(i_category_id is not null and i_brand_id is not null and i_class_id is not null)
Please refer to the previous TableScan [TS_6]
<-Reducer 27 [SIMPLE_EDGE]
SHUFFLE [RS_163]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_1132] (rows=14736682 width=4)
- Conds:RS_1396._col0=RS_1293._col0(Inner),Output:["_col1"]
+ Merge Join Operator [MERGEJOIN_1192] (rows=14736682 width=4)
+ Conds:RS_1456._col0=RS_1353._col0(Inner),Output:["_col1"]
<-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1293]
+ SHUFFLE [RS_1353]
PartitionCols:_col0
- Select Operator [SEL_1286] (rows=1957 width=4)
+ Select Operator [SEL_1346] (rows=1957 width=4)
Output:["_col0"]
- Filter Operator [FIL_1284] (rows=1957 width=8)
+ Filter Operator [FIL_1344] (rows=1957 width=8)
predicate:d_year BETWEEN 1999 AND 2001
Please refer to the previous TableScan [TS_3]
<-Map 63 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1396]
+ SHUFFLE [RS_1456]
PartitionCols:_col0
- Select Operator [SEL_1394] (rows=550076554 width=7)
+ Select Operator [SEL_1454] (rows=550076554 width=7)
Output:["_col0","_col1"]
- Filter Operator [FIL_1392] (rows=550076554 width=7)
+ Filter Operator [FIL_1452] (rows=550076554 width=7)
predicate:ss_sold_date_sk is not null
TableScan [TS_9] (rows=575995635 width=7)
default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_item_sk","ss_quantity","ss_list_price"]
<-Reducer 43 [CONTAINS] vectorized
- Reduce Output Operator [RS_1419]
+ Reduce Output Operator [RS_1479]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_1418] (rows=121728 width=19)
+ Group By Operator [GBY_1478] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(_col3)"],keys:_col0, _col1, _col2
- Group By Operator [GBY_1417] (rows=121728 width=19)
+ Group By Operator [GBY_1477] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 42 [SIMPLE_EDGE]
SHUFFLE [RS_188]
PartitionCols:_col0, _col1, _col2
Group By Operator [GBY_187] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count()"],keys:_col4, _col5, _col6
- Merge Join Operator [MERGEJOIN_1148] (rows=7620440 width=11)
- Conds:RS_183._col1=RS_1340._col0(Inner),Output:["_col4","_col5","_col6"]
+ Merge Join Operator [MERGEJOIN_1208] (rows=7620440 width=11)
+ Conds:RS_183._col1=RS_1400._col0(Inner),Output:["_col4","_col5","_col6"]
<-Map 62 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1340]
+ SHUFFLE [RS_1400]
PartitionCols:_col0
- Select Operator [SEL_1329] (rows=458612 width=15)
+ Select Operator [SEL_1389] (rows=458612 width=15)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_1320] (rows=458612 width=15)
+ Filter Operator [FIL_1380] (rows=458612 width=15)
predicate:(i_category_id is not null and i_brand_id is not null and i_class_id is not null)
Please refer to the previous TableScan [TS_6]
<-Reducer 39 [SIMPLE_EDGE]
SHUFFLE [RS_183]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_1134] (rows=7676736 width=4)
- Conds:RS_1413._col0=RS_1294._col0(Inner),Output:["_col1"]
+ Merge Join Operator [MERGEJOIN_1194] (rows=7676736 width=4)
+ Conds:RS_1473._col0=RS_1354._col0(Inner),Output:["_col1"]
<-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1294]
+ SHUFFLE [RS_1354]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1286]
+ Please refer to the previous Select Operator [SEL_1346]
<-Map 64 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1413]
+ SHUFFLE [RS_1473]
PartitionCols:_col0
- Select Operator [SEL_1412] (rows=286549727 width=7)
+ Select Operator [SEL_1472] (rows=286549727 width=7)
Output:["_col0","_col1"]
- Filter Operator [FIL_1411] (rows=286549727 width=7)
+ Filter Operator [FIL_1471] (rows=286549727 width=7)
predicate:(cs_sold_date_sk is not null and cs_sold_date_sk BETWEEN DynamicValue(RS_39_d2_d_date_sk_min) AND DynamicValue(RS_39_d2_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_39_d2_d_date_sk_bloom_filter)))
TableScan [TS_29] (rows=287989836 width=7)
default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["cs_sold_date_sk","cs_item_sk"]
<-Reducer 45 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_1410]
- Group By Operator [GBY_1409] (rows=1 width=12)
+ BROADCAST [RS_1470]
+ Group By Operator [GBY_1469] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 11 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1309]
- Group By Operator [GBY_1304] (rows=1 width=12)
+ SHUFFLE [RS_1369]
+ Group By Operator [GBY_1364] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_1295] (rows=1957 width=4)
+ Select Operator [SEL_1355] (rows=1957 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_1286]
+ Please refer to the previous Select Operator [SEL_1346]
<-Reducer 50 [CONTAINS] vectorized
- Reduce Output Operator [RS_1433]
+ Reduce Output Operator [RS_1493]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_1432] (rows=121728 width=19)
+ Group By Operator [GBY_1492] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(_col3)"],keys:_col0, _col1, _col2
- Group By Operator [GBY_1431] (rows=121728 width=19)
+ Group By Operator [GBY_1491] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 49 [SIMPLE_EDGE]
SHUFFLE [RS_209]
PartitionCols:_col0, _col1, _col2
Group By Operator [GBY_208] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count()"],keys:_col4, _col5, _col6
- Merge Join Operator [MERGEJOIN_1150] (rows=3828623 width=11)
- Conds:RS_204._col1=RS_1341._col0(Inner),Output:["_col4","_col5","_col6"]
+ Merge Join Operator [MERGEJOIN_1210] (rows=3828623 width=11)
+ Conds:RS_204._col1=RS_1401._col0(Inner),Output:["_col4","_col5","_col6"]
<-Map 62 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1341]
+ SHUFFLE [RS_1401]
PartitionCols:_col0
- Select Operator [SEL_1330] (rows=458612 width=15)
+ Select Operator [SEL_1390] (rows=458612 width=15)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_1321] (rows=458612 width=15)
+ Filter Operator [FIL_1381] (rows=458612 width=15)
predicate:(i_category_id is not null and i_brand_id is not null and i_class_id is not null)
Please refer to the previous TableScan [TS_6]
<-Reducer 46 [SIMPLE_EDGE]
SHUFFLE [RS_204]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_1136] (rows=3856907 width=4)
- Conds:RS_1427._col0=RS_1296._col0(Inner),Output:["_col1"]
+ Merge Join Operator [MERGEJOIN_1196] (rows=3856907 width=4)
+ Conds:RS_1487._col0=RS_1356._col0(Inner),Output:["_col1"]
<-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1296]
+ SHUFFLE [RS_1356]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1286]
+ Please refer to the previous Select Operator [SEL_1346]
<-Map 65 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1427]
+ SHUFFLE [RS_1487]
PartitionCols:_col0
- Select Operator [SEL_1426] (rows=143966864 width=7)
+ Select Operator [SEL_1486] (rows=143966864 width=7)
Output:["_col0","_col1"]
- Filter Operator [FIL_1425] (rows=143966864 width=7)
+ Filter Operator [FIL_1485] (rows=143966864 width=7)
predicate:(ws_sold_date_sk is not null and ws_sold_date_sk BETWEEN DynamicValue(RS_60_d3_d_date_sk_min) AND DynamicValue(RS_60_d3_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_60_d3_d_date_sk_bloom_filter)))
TableScan [TS_50] (rows=144002668 width=7)
default@web_sales,web_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ws_sold_date_sk","ws_item_sk"]
<-Reducer 52 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_1424]
- Group By Operator [GBY_1423] (rows=1 width=12)
+ BROADCAST [RS_1484]
+ Group By Operator [GBY_1483] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 11 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1310]
- Group By Operator [GBY_1305] (rows=1 width=12)
+ SHUFFLE [RS_1370]
+ Group By Operator [GBY_1365] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_1297] (rows=1957 width=4)
+ Select Operator [SEL_1357] (rows=1957 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_1286]
+ Please refer to the previous Select Operator [SEL_1346]
<-Reducer 58 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1375]
- Select Operator [SEL_1374] (rows=1 width=112)
+ PARTITION_ONLY_SHUFFLE [RS_1435]
+ Select Operator [SEL_1434] (rows=1 width=112)
Output:["_col0"]
- Filter Operator [FIL_1373] (rows=1 width=120)
+ Filter Operator [FIL_1433] (rows=1 width=120)
predicate:CAST( (_col0 / _col1) AS decimal(22,6)) is not null
- Group By Operator [GBY_1372] (rows=1 width=120)
+ Group By Operator [GBY_1432] (rows=1 width=120)
Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"]
<-Union 57 [CUSTOM_SIMPLE_EDGE]
<-Reducer 56 [CONTAINS]
- Reduce Output Operator [RS_1240]
- Group By Operator [GBY_1239] (rows=1 width=120)
+ Reduce Output Operator [RS_1300]
+ Group By Operator [GBY_1299] (rows=1 width=120)
Output:["_col0","_col1"],aggregations:["sum(_col0)","count(_col0)"]
- Select Operator [SEL_1238] (rows=26270325 width=44)
+ Select Operator [SEL_1298] (rows=26270325 width=44)
Output:["_col0"]
- Select Operator [SEL_1236] (rows=14736682 width=0)
+ Select Operator [SEL_1296] (rows=14736682 width=0)
Output:["_col0","_col1"]
- Merge Join Operator [MERGEJOIN_1235] (rows=14736682 width=0)
- Conds:RS_1398._col0=RS_1299._col0(Inner),Output:["_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_1295] (rows=14736682 width=0)
+ Conds:RS_1458._col0=RS_1359._col0(Inner),Output:["_col1","_col2"]
<-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1299]
+ SHUFFLE [RS_1359]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1286]
+ Please refer to the previous Select Operator [SEL_1346]
<-Map 63 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1398]
+ SHUFFLE [RS_1458]
PartitionCols:_col0
- Select Operator [SEL_1395] (rows=550076554 width=114)
+ Select Operator [SEL_1455] (rows=550076554 width=114)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_1393] (rows=550076554 width=114)
+ Filter Operator [FIL_1453] (rows=550076554 width=114)
predicate:(ss_sold_date_sk is not null and ss_sold_date_sk is not null)
Please refer to the previous TableScan [TS_9]
<-Reducer 68 [CONTAINS]
- Reduce Output Operator [RS_1258]
- Group By Operator [GBY_1257] (rows=1 width=120)
+ Reduce Output Operator [RS_1318]
+ Group By Operator [GBY_1317] (rows=1 width=120)
Output:["_col0","_col1"],aggregations:["sum(_col0)","count(_col0)"]
- Select Operator [SEL_1256] (rows=26270325 width=44)
+ Select Operator [SEL_1316] (rows=26270325 width=44)
Output:["_col0"]
- Select Operator [SEL_1254] (rows=7676736 width=94)
+ Select Operator [SEL_1314] (rows=7676736 width=94)
Output:["_col0","_col1"]
- Merge Join Operator [MERGEJOIN_1253] (rows=7676736 width=94)
- Conds:RS_1450._col0=RS_1441._col0(Inner),Output:["_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_1313] (rows=7676736 width=94)
+ Conds:RS_1510._col0=RS_1501._col0(Inner),Output:["_col1","_col2"]
<-Map 70 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1441]
+ PARTITION_ONLY_SHUFFLE [RS_1501]
PartitionCols:_col0
- Select Operator [SEL_1438] (rows=1957 width=4)
+ Select Operator [SEL_1498] (rows=1957 width=4)
Output:["_col0"]
- Filter Operator [FIL_1437] (rows=1957 width=8)
+ Filter Operator [FIL_1497] (rows=1957 width=8)
predicate:d_year BETWEEN 1998 AND 2000
TableScan [TS_111] (rows=73049 width=8)
default@date_dim,date_dim,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_year"]
<-Map 66 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1450]
+ SHUFFLE [RS_1510]
PartitionCols:_col0
- Select Operator [SEL_1448] (rows=286549727 width=119)
+ Select Operator [SEL_1508] (rows=286549727 width=119)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_1447] (rows=286549727 width=119)
+ Filter Operator [FIL_1507] (rows=286549727 width=119)
predicate:(cs_sold_date_sk is not null and cs_sold_date_sk BETWEEN DynamicValue(RS_115_date_dim_d_date_sk_min) AND DynamicValue(RS_115_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_115_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_108] (rows=287989836 width=119)
default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["cs_sold_date_sk","cs_quantity","cs_list_price"]
<-Reducer 71 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_1446]
- Group By Operator [GBY_1445] (rows=1 width=12)
+ BROADCAST [RS_1506]
+ Group By Operator [GBY_1505] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 70 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1444]
- Group By Operator [GBY_1443] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_1504]
+ Group By Operator [GBY_1503] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_1440] (rows=1957 width=4)
+ Select Operator [SEL_1500] (rows=1957 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_1438]
+ Please refer to the previous Select Operator [SEL_1498]
<-Reducer 74 [CONTAINS]
- Reduce Output Operator [RS_1276]
- Group By Operator [GBY_1275] (rows=1 width=120)
+ Reduce Output Operator [RS_1336]
+ Group By Operator [GBY_1335] (rows=1 width=120)
Output:["_col0","_col1"],aggregations:["sum(_col0)","count(_col0)"]
- Select Operator [SEL_1274] (rows=26270325 width=44)
+ Select Operator [SEL_1334] (rows=26270325 width=44)
Output:["_col0"]
- Select Operator [SEL_1272] (rows=3856907 width=114)
+ Select Operator [SEL_1332] (rows=3856907 width=114)
Output:["_col0","_col1"]
- Merge Join Operator [MERGEJOIN_1271] (rows=3856907 width=114)
- Conds:RS_1465._col0=RS_1456._col0(Inner),Output:["_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_1331] (rows=3856907 width=114)
+ Conds:RS_1525._col0=RS_1516._col0(Inner),Output:["_col1","_col2"]
<-Map 76 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1456]
+ PARTITION_ONLY_SHUFFLE [RS_1516]
PartitionCols:_col0
- Select Operator [SEL_1453] (rows=1957 width=4)
+ Select Operator [SEL_1513] (rows=1957 width=4)
Output:["_col0"]
- Filter Operator [FIL_1452] (rows=1957 width=8)
+ Filter Operator [FIL_1512] (rows=1957 width=8)
predicate:d_year BETWEEN 1998 AND 2000
TableScan [TS_122] (rows=73049 width=8)
default@date_dim,date_dim,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_year"]
<-Map 72 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1465]
+ SHUFFLE [RS_1525]
PartitionCols:_col0
- Select Operator [SEL_1463] (rows=143966864 width=119)
+ Select Operator [SEL_1523] (rows=143966864 width=119)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_1462] (rows=143966864 width=119)
+ Filter Operator [FIL_1522] (rows=143966864 width=119)
predicate:(ws_sold_date_sk is not null and ws_sold_date_sk BETWEEN DynamicValue(RS_126_date_dim_d_date_sk_min) AND DynamicValue(RS_126_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_126_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_119] (rows=144002668 width=119)
default@web_sales,web_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ws_sold_date_sk","ws_quantity","ws_list_price"]
<-Reducer 77 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_1461]
- Group By Operator [GBY_1460] (rows=1 width=12)
+ BROADCAST [RS_1521]
+ Group By Operator [GBY_1520] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 76 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1459]
- Group By Operator [GBY_1458] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_1519]
+ Group By Operator [GBY_1518] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_1455] (rows=1957 width=4)
+ Select Operator [SEL_1515] (rows=1957 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_1453]
+ Please refer to the previous Select Operator [SEL_1513]
<-Reducer 25 [CONTAINS]
- Reduce Output Operator [RS_1192]
+ Reduce Output Operator [RS_1252]
PartitionCols:_col0, _col1, _col2, _col3, _col4
- Group By Operator [GBY_1191] (rows=205 width=228)
+ Group By Operator [GBY_1251] (rows=205 width=228)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["sum(_col4)","sum(_col5)"],keys:_col0, _col1, _col2, _col3, 0L
- Top N Key Operator [TNK_1190] (rows=82 width=220)
+ Top N Key Operator [TNK_1250] (rows=82 width=220)
keys:_col0, _col1, _col2, _col3,top n:100
- Select Operator [SEL_1188] (rows=13 width=218)
+ Select Operator [SEL_1248] (rows=13 width=218)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_1187] (rows=13 width=243)
+ Filter Operator [FIL_1247] (rows=13 width=243)
predicate:(_col3 > _col5)
- Merge Join Operator [MERGEJOIN_1186] (rows=40 width=242)
+ Merge Join Operator [MERGEJOIN_1246] (rows=40 width=242)
Conds:(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
<-Reducer 24 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1387]
- Filter Operator [FIL_1386] (rows=40 width=130)
+ PARTITION_ONLY_SHUFFLE [RS_1447]
+ Filter Operator [FIL_1446] (rows=40 width=130)
predicate:_col3 is not null
- Group By Operator [GBY_1385] (rows=40 width=130)
+ Group By Operator [GBY_1445] (rows=40 width=130)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 23 [SIMPLE_EDGE]
SHUFFLE [RS_379]
@@ -685,248 +685,248 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col3)","count()"],keys:_col0, _col1, _col2
Select Operator [SEL_376] (rows=6181 width=9)
Output:["_col0","_col1","_col2","_col3"]
- Merge Join Operator [MERGEJOIN_1165] (rows=6181 width=9)
- Conds:RS_373._col1=RS_1333._col0(Inner),Output:["_col2","_col3","_col13","_col14","_col15"]
+ Merge Join Operator [MERGEJOIN_1225] (rows=6181 width=9)
+ Conds:RS_373._col1=RS_1393._col0(Inner),Output:["_col2","_col3","_col13","_col14","_col15"]
<-Map 62 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1333]
+ SHUFFLE [RS_1393]
PartitionCols:_col0
- Select Operator [SEL_1324] (rows=462000 width=15)
+ Select Operator [SEL_1384] (rows=462000 width=15)
Output:["_col0","_col1","_col2","_col3"]
Please refer to the previous TableScan [TS_6]
<-Reducer 22 [SIMPLE_EDGE]
SHUFFLE [RS_373]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_1164] (rows=6181 width=4)
- Conds:RS_370._col6, _col7, _col8=RS_1384._col0, _col1, _col2(Inner),Output:["_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_1224] (rows=6181 width=4)
+ Conds:RS_370._col6, _col7, _col8=RS_1444._col0, _col1, _col2(Inner),Output:["_col1","_col2","_col3"]
<-Reducer 21 [SIMPLE_EDGE]
SHUFFLE [RS_370]
PartitionCols:_col6, _col7, _col8
- Merge Join Operator [MERGEJOIN_1157] (rows=3913176 width=130)
- Conds:RS_367._col1=RS_1342._col0(Inner),Output:["_col1","_col2","_col3","_col6","_col7","_col8"]
+ Merge Join Operator [MERGEJOIN_1217] (rows=3913176 width=130)
+ Conds:RS_367._col1=RS_1402._col0(Inner),Output:["_col1","_col2","_col3","_col6","_col7","_col8"]
<-Map 62 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1342]
+ SHUFFLE [RS_1402]
PartitionCols:_col0
- Select Operator [SEL_1332] (rows=458612 width=15)
+ Select Operator [SEL_1392] (rows=458612 width=15)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_1323] (rows=458612 width=15)
+ Filter Operator [FIL_1383] (rows=458612 width=15)
predicate:(i_category_id is not null and i_brand_id is not null and i_class_id is not null)
Please refer to the previous TableScan [TS_6]
<-Reducer 20 [SIMPLE_EDGE]
SHUFFLE [RS_367]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_1156] (rows=3942084 width=118)
- Conds:RS_1380._col0=RS_1291._col0(Inner),Output:["_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_1216] (rows=3942084 width=118)
+ Conds:RS_1440._col0=RS_1351._col0(Inner),Output:["_col1","_col2","_col3"]
<-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1291]
+ SHUFFLE [RS_1351]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1285]
+ Please refer to the previous Select Operator [SEL_1345]
<-Map 79 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1380]
+ SHUFFLE [RS_1440]
PartitionCols:_col0
- Select Operator [SEL_1379] (rows=143966864 width=123)
+ Select Operator [SEL_1439] (rows=143966864 width=123)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_1378] (rows=143966864 width=123)
+ Filter Operator [FIL_1438] (rows=143966864 width=123)
predicate:(ws_sold_date_sk is not null and ws_sold_date_sk BETWEEN DynamicValue(RS_365_date_dim_d_date_sk_min) AND DynamicValue(RS_365_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_365_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_285] (rows=144002668 width=123)
default@web_sales,web_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ws_sold_date_sk","ws_item_sk","ws_quantity","ws_list_price"]
<-Reducer 26 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_1377]
- Group By Operator [GBY_1376] (rows=1 width=12)
+ BROADCAST [RS_1437]
+ Group By Operator [GBY_1436] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 11 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1308]
- Group By Operator [GBY_1303] (rows=1 width=12)
+ SHUFFLE [RS_1368]
+ Group By Operator [GBY_1363] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_1292] (rows=50 width=4)
+ Select Operator [SEL_1352] (rows=50 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_1285]
+ Please refer to the previous Select Operator [SEL_1345]
<-Reducer 38 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1384]
+ SHUFFLE [RS_1444]
PartitionCols:_col0, _col1, _col2
- Select Operator [SEL_1383] (rows=1 width=12)
+ Select Operator [SEL_1443] (rows=1 width=12)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_1382] (rows=1 width=20)
+ Filter Operator [FIL_1442] (rows=1 width=20)
predicate:(_col3 = 3L)
- Group By Operator [GBY_1381] (rows=121728 width=19)
+ Group By Operator [GBY_1441] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Union 37 [SIMPLE_EDGE]
<-Reducer 36 [CONTAINS] vectorized
- Reduce Output Operator [RS_1408]
+ Reduce Output Operator [RS_1468]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_1407] (rows=121728 width=19)
+ Group By Operator [GBY_1467] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(_col3)"],keys:_col0, _col1, _col2
- Group By Operator [GBY_1406] (rows=121728 width=19)
+ Group By Operator [GBY_1466] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 32 [SIMPLE_EDGE]
SHUFFLE [RS_311]
PartitionCols:_col0, _col1, _col2
Please refer to the previous Group By Operator [GBY_167]
<-Reducer 44 [CONTAINS] vectorized
- Reduce Output Operator [RS_1422]
+ Reduce Output Operator [RS_1482]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_1421] (rows=121728 width=19)
+ Group By Operator [GBY_1481] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(_col3)"],keys:_col0, _col1, _col2
- Group By Operator [GBY_1420] (rows=121728 width=19)
+ Group By Operator [GBY_1480] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 42 [SIMPLE_EDGE]
SHUFFLE [RS_331]
PartitionCols:_col0, _col1, _col2
Please refer to the previous Group By Operator [GBY_187]
<-Reducer 51 [CONTAINS] vectorized
- Reduce Output Operator [RS_1436]
+ Reduce Output Operator [RS_1496]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_1435] (rows=121728 width=19)
+ Group By Operator [GBY_1495] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(_col3)"],keys:_col0, _col1, _col2
- Group By Operator [GBY_1434] (rows=121728 width=19)
+ Group By Operator [GBY_1494] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 49 [SIMPLE_EDGE]
SHUFFLE [RS_352]
PartitionCols:_col0, _col1, _col2
Please refer to the previous Group By Operator [GBY_208]
<-Reducer 61 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1391]
- Select Operator [SEL_1390] (rows=1 width=112)
+ PARTITION_ONLY_SHUFFLE [RS_1451]
+ Select Operator [SEL_1450] (rows=1 width=112)
Output:["_col0"]
- Filter Operator [FIL_1389] (rows=1 width=120)
+ Filter Operator [FIL_1449] (rows=1 width=120)
predicate:CAST( (_col0 / _col1) AS decimal(22,6)) is not null
- Group By Operator [GBY_1388] (rows=1 width=120)
+ Group By Operator [GBY_1448] (rows=1 width=120)
Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"]
<-Union 60 [CUSTOM_SIMPLE_EDGE]
<-Reducer 59 [CONTAINS]
- Reduce Output Operator [RS_1246]
- Group By Operator [GBY_1245] (rows=1 width=120)
+ Reduce Output Operator [RS_1306]
+ Group By Operator [GBY_1305] (rows=1 width=120)
Output:["_col0","_col1"],aggregations:["sum(_col0)","count(_col0)"]
- Select Operator [SEL_1244] (rows=26270325 width=44)
+ Select Operator [SEL_1304] (rows=26270325 width=44)
Output:["_col0"]
- Select Operator [SEL_1242] (rows=14736682 width=0)
+ Select Operator [SEL_1302] (rows=14736682 width=0)
Output:["_col0","_col1"]
- Merge Join Operator [MERGEJOIN_1241] (rows=14736682 width=0)
- Conds:RS_1399._col0=RS_1300._col0(Inner),Output:["_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_1301] (rows=14736682 width=0)
+ Conds:RS_1459._col0=RS_1360._col0(Inner),Output:["_col1","_col2"]
<-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1300]
+ SHUFFLE [RS_1360]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1286]
+ Please refer to the previous Select Operator [SEL_1346]
<-Map 63 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1399]
+ SHUFFLE [RS_1459]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1395]
+ Please refer to the previous Select Operator [SEL_1455]
<-Reducer 69 [CONTAINS]
- Reduce Output Operator [RS_1264]
- Group By Operator [GBY_1263] (rows=1 width=120)
+ Reduce Output Operator [RS_1324]
+ Group By Operator [GBY_1323] (rows=1 width=120)
Output:["_col0","_col1"],aggregations:["sum(_col0)","count(_col0)"]
- Select Operator [SEL_1262] (rows=26270325 width=44)
+ Select Operator [SEL_1322] (rows=26270325 width=44)
Output:["_col0"]
- Select Operator [SEL_1260] (rows=7676736 width=94)
+ Select Operator [SEL_1320] (rows=7676736 width=94)
Output:["_col0","_col1"]
- Merge Join Operator [MERGEJOIN_1259] (rows=7676736 width=94)
- Conds:RS_1451._col0=RS_1442._col0(Inner),Output:["_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_1319] (rows=7676736 width=94)
+ Conds:RS_1511._col0=RS_1502._col0(Inner),Output:["_col1","_col2"]
<-Map 70 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1442]
+ PARTITION_ONLY_SHUFFLE [RS_1502]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1438]
+ Please refer to the previous Select Operator [SEL_1498]
<-Map 66 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1451]
+ SHUFFLE [RS_1511]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1448]
+ Please refer to the previous Select Operator [SEL_1508]
<-Reducer 75 [CONTAINS]
- Reduce Output Operator [RS_1282]
- Group By Operator [GBY_1281] (rows=1 width=120)
+ Reduce Output Operator [RS_1342]
+ Group By Operator [GBY_1341] (rows=1 width=120)
Output:["_col0","_col1"],aggregations:["sum(_col0)","count(_col0)"]
- Select Operator [SEL_1280] (rows=26270325 width=44)
+ Select Operator [SEL_1340] (rows=26270325 width=44)
Output:["_col0"]
- Select Operator [SEL_1278] (rows=3856907 width=114)
+ Select Operator [SEL_1338] (rows=3856907 width=114)
Output:["_col0","_col1"]
- Merge Join Operator [MERGEJOIN_1277] (rows=3856907 width=114)
- Conds:RS_1466._col0=RS_1457._col0(Inner),Output:["_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_1337] (rows=3856907 width=114)
+ Conds:RS_1526._col0=RS_1517._col0(Inner),Output:["_col1","_col2"]
<-Map 76 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1457]
+ PARTITION_ONLY_SHUFFLE [RS_1517]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1453]
+ Please refer to the previous Select Operator [SEL_1513]
<-Map 72 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1466]
+ SHUFFLE [RS_1526]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1463]
+ Please refer to the previous Select Operator [SEL_1523]
<-Reducer 7 [CONTAINS]
- Reduce Output Operator [RS_1178]
+ Reduce Output Operator [RS_1238]
PartitionCols:_col0, _col1, _col2, _col3, _col4
- Group By Operator [GBY_1177] (rows=205 width=228)
+ Group By Operator [GBY_1237] (rows=205 width=228)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["sum(_col4)","sum(_col5)"],keys:_col0, _col1, _col2, _col3, 0L
- Top N Key Operator [TNK_1176] (rows=82 width=220)
+ Top N Key Operator [TNK_1236] (rows=82 width=220)
keys:_col0, _col1, _col2, _col3,top n:100
- Select Operator [SEL_1174] (rows=45 width=220)
+ Select Operator [SEL_1234] (rows=45 width=220)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_1173] (rows=45 width=243)
+ Filter Operator [FIL_1233] (rows=45 width=243)
predicate:(_col3 > _col5)
- Merge Join Operator [MERGEJOIN_1172] (rows=136 width=243)
+ Merge Join Operator [MERGEJOIN_1232] (rows=136 width=243)
Conds:(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
<-Reducer 55 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1353]
- Select Operator [SEL_1352] (rows=1 width=112)
+ PARTITION_ONLY_SHUFFLE [RS_1413]
+ Select Operator [SEL_1412] (rows=1 width=112)
Output:["_col0"]
- Filter Operator [FIL_1351] (rows=1 width=120)
+ Filter Operator [FIL_1411] (rows=1 width=120)
predicate:CAST( (_col0 / _col1) AS decimal(22,6)) is not null
- Group By Operator [GBY_1350] (rows=1 width=120)
+ Group By Operator [GBY_1410] (rows=1 width=120)
Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"]
<-Union 54 [CUSTOM_SIMPLE_EDGE]
<-Reducer 53 [CONTAINS]
- Reduce Output Operator [RS_1234]
- Group By Operator [GBY_1233] (rows=1 width=120)
+ Reduce Output Operator [RS_1294]
+ Group By Operator [GBY_1293] (rows=1 width=120)
Output:["_col0","_col1"],aggregations:["sum(_col0)","count(_col0)"]
- Select Operator [SEL_1232] (rows=26270325 width=44)
+ Select Operator [SEL_1292] (rows=26270325 width=44)
Output:["_col0"]
- Select Operator [SEL_1230] (rows=14736682 width=0)
+ Select Operator [SEL_1290] (rows=14736682 width=0)
Output:["_col0","_col1"]
- Merge Join Operator [MERGEJOIN_1229] (rows=14736682 width=0)
- Conds:RS_1397._col0=RS_1298._col0(Inner),Output:["_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_1289] (rows=14736682 width=0)
+ Conds:RS_1457._col0=RS_1358._col0(Inner),Output:["_col1","_col2"]
<-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1298]
+ SHUFFLE [RS_1358]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1286]
+ Please refer to the previous Select Operator [SEL_1346]
<-Map 63 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1397]
+ SHUFFLE [RS_1457]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1395]
+ Please refer to the previous Select Operator [SEL_1455]
<-Reducer 67 [CONTAINS]
- Reduce Output Operator [RS_1252]
- Group By Operator [GBY_1251] (rows=1 width=120)
+ Reduce Output Operator [RS_1312]
+ Group By Operator [GBY_1311] (rows=1 width=120)
Output:["_col0","_col1"],aggregations:["sum(_col0)","count(_col0)"]
- Select Operator [SEL_1250] (rows=26270325 width=44)
+ Select Operator [SEL_1310] (rows=26270325 width=44)
Output:["_col0"]
- Select Operator [SEL_1248] (rows=7676736 width=94)
+ Select Operator [SEL_1308] (rows=7676736 width=94)
Output:["_col0","_col1"]
- Merge Join Operator [MERGEJOIN_1247] (rows=7676736 width=94)
- Conds:RS_1449._col0=RS_1439._col0(Inner),Output:["_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_1307] (rows=7676736 width=94)
+ Conds:RS_1509._col0=RS_1499._col0(Inner),Output:["_col1","_col2"]
<-Map 70 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1439]
+ PARTITION_ONLY_SHUFFLE [RS_1499]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1438]
+ Please refer to the previous Select Operator [SEL_1498]
<-Map 66 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1449]
+ SHUFFLE [RS_1509]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1448]
+ Please refer to the previous Select Operator [SEL_1508]
<-Reducer 73 [CONTAINS]
- Reduce Output Operator [RS_1270]
- Group By Operator [GBY_1269] (rows=1 width=120)
+ Reduce Output Operator [RS_1330]
+ Group By Operator [GBY_1329] (rows=1 width=120)
Output:["_col0","_col1"],aggregations:["sum(_col0)","count(_col0)"]
- Select Operator [SEL_1268] (rows=26270325 width=44)
+ Select Operator [SEL_1328] (rows=26270325 width=44)
Output:["_col0"]
- Select Operator [SEL_1266] (rows=3856907 width=114)
+ Select Operator [SEL_1326] (rows=3856907 width=114)
Output:["_col0","_col1"]
- Merge Join Operator [MERGEJOIN_1265] (rows=3856907 width=114)
- Conds:RS_1464._col0=RS_1454._col0(Inner),Output:["_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_1325] (rows=3856907 width=114)
+ Conds:RS_1524._col0=RS_1514._col0(Inner),Output:["_col1","_col2"]
<-Map 76 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1454]
+ PARTITION_ONLY_SHUFFLE [RS_1514]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1453]
+ Please refer to the previous Select Operator [SEL_1513]
<-Map 72 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1464]
+ SHUFFLE [RS_1524]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1463]
+ Please refer to the previous Select Operator [SEL_1523]
<-Reducer 6 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1349]
- Filter Operator [FIL_1348] (rows=136 width=131)
+ PARTITION_ONLY_SHUFFLE [RS_1409]
+ Filter Operator [FIL_1408] (rows=136 width=131)
predicate:_col3 is not null
- Group By Operator [GBY_1347] (rows=136 width=131)
+ Group By Operator [GBY_1407] (rows=136 width=131)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_94]
@@ -935,135 +935,135 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col3)","count()"],keys:_col0, _col1, _col2
Select Operator [SEL_91] (rows=23620 width=11)
Output:["_col0","_col1","_col2","_col3"]
- Merge Join Operator [MERGEJOIN_1139] (rows=23620 width=11)
- Conds:RS_88._col1=RS_1326._col0(Inner),Output:["_col2","_col3","_col13","_col14","_col15"]
+ Merge Join Operator [MERGEJOIN_1199] (rows=23620 width=11)
+ Conds:RS_88._col1=RS_1386._col0(Inner),Output:["_col2","_col3","_col13","_col14","_col15"]
<-Map 62 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1326]
+ SHUFFLE [RS_1386]
PartitionCols:_col0
- Select Operator [SEL_1317] (rows=462000 width=15)
+ Select Operator [SEL_1377] (rows=462000 width=15)
Output:["_col0","_col1","_col2","_col3"]
Please refer to the previous TableScan [TS_6]
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_88]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_1138] (rows=23620 width=4)
- Conds:RS_85._col6, _col7, _col8=RS_1346._col0, _col1, _col2(Inner),Output:["_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_1198] (rows=23620 width=4)
+ Conds:RS_85._col6, _col7, _col8=RS_1406._col0, _col1, _col2(Inner),Output:["_col1","_col2","_col3"]
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_85]
PartitionCols:_col6, _col7, _col8
- Merge Join Operator [MERGEJOIN_1131] (rows=14951676 width=15)
- Conds:RS_82._col1=RS_1334._col0(Inner),Output:["_col1","_col2","_col3","_col6","_col7","_col8"]
+ Merge Join Operator [MERGEJOIN_1191] (rows=14951676 width=15)
+ Conds:RS_82._col1=RS_1394._col0(Inner),Output:["_col1","_col2","_col3","_col6","_col7","_col8"]
<-Map 62 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1334]
+ SHUFFLE [RS_1394]
PartitionCols:_col0
- Select Operator [SEL_1325] (rows=458612 width=15)
+ Select Operator [SEL_1385] (rows=458612 width=15)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_1316] (rows=458612 width=15)
+ Filter Operator [FIL_1376] (rows=458612 width=15)
predicate:(i_category_id is not null and i_brand_id is not null and i_class_id is not null)
Please refer to the previous TableScan [TS_6]
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_82]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_1130] (rows=15062131 width=4)
- Conds:RS_1315._col0=RS_1287._col0(Inner),Output:["_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_1190] (rows=15062131 width=4)
+ Conds:RS_1375._col0=RS_1347._col0(Inner),Output:["_col1","_col2","_col3"]
<-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1287]
+ SHUFFLE [RS_1347]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1285]
+ Please refer to the previous Select Operator [SEL_1345]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1315]
+ SHUFFLE [RS_1375]
PartitionCols:_col0
- Select Operator [SEL_1314] (rows=550076554 width=118)
+ Select Operator [SEL_1374] (rows=550076554 width=118)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_1313] (rows=550076554 width=118)
+ Filter Operator [FIL_1373] (rows=550076554 width=118)
predicate:(ss_sold_date_sk is not null and ss_sold_date_sk BETWEEN DynamicValue(RS_80_date_dim_d_date_sk_min) AND DynamicValue(RS_80_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_80_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_0] (rows=575995635 width=118)
default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_item_sk","ss_quantity","ss_list_price"]
<-Reducer 12 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_1312]
- Group By Operator [GBY_1311] (rows=1 width=12)
+ BROADCAST [RS_1372]
+ Group By Operator [GBY_1371] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 11 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1306]
- Group By Operator [GBY_1301] (rows=1 width=12)
+ SHUFFLE [RS_1366]
+ Group By Operator [GBY_1361] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_1288] (rows=50 width=4)
+ Select Operator [SEL_1348] (rows=50 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_1285]
+ Please refer to the previous Select Operator [SEL_1345]
<-Reducer 31 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1346]
+ SHUFFLE [RS_1406]
PartitionCols:_col0, _col1, _col2
- Select Operator [SEL_1345] (rows=1 width=12)
+ Select Operator [SEL_1405] (rows=1 width=12)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_1344] (rows=1 width=20)
+ Filter Operator [FIL_1404] (rows=1 width=20)
predicate:(_col3 = 3L)
- Group By Operator [GBY_1343] (rows=121728 width=19)
+ Group By Operator [GBY_1403] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Union 30 [SIMPLE_EDGE]
<-Reducer 29 [CONTAINS] vectorized
- Reduce Output Operator [RS_1402]
+ Reduce Output Operator [RS_1462]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_1401] (rows=121728 width=19)
+ Group By Operator [GBY_1461] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(_col3)"],keys:_col0, _col1, _col2
- Group By Operator [GBY_1400] (rows=121728 width=19)
+ Group By Operator [GBY_1460] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 28 [SIMPLE_EDGE]
SHUFFLE [RS_26]
PartitionCols:_col0, _col1, _col2
Group By Operator [GBY_25] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count()"],keys:_col4, _col5, _col6
- Merge Join Operator [MERGEJOIN_1133] (rows=14628613 width=11)
- Conds:RS_21._col1=RS_1335._col0(Inner),Output:["_col4","_col5","_col6"]
+ Merge Join Operator [MERGEJOIN_1193] (rows=14628613 width=11)
+ Conds:RS_21._col1=RS_1395._col0(Inner),Output:["_col4","_col5","_col6"]
<-Map 62 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1335]
+ SHUFFLE [RS_1395]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1325]
+ Please refer to the previous Select Operator [SEL_1385]
<-Reducer 27 [SIMPLE_EDGE]
SHUFFLE [RS_21]
PartitionCols:_col1
- Please refer to the previous Merge Join Operator [MERGEJOIN_1132]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_1192]
<-Reducer 41 [CONTAINS] vectorized
- Reduce Output Operator [RS_1416]
+ Reduce Output Operator [RS_1476]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_1415] (rows=121728 width=19)
+ Group By Operator [GBY_1475] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(_col3)"],keys:_col0, _col1, _col2
- Group By Operator [GBY_1414] (rows=121728 width=19)
+ Group By Operator [GBY_1474] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 40 [SIMPLE_EDGE]
SHUFFLE [RS_46]
PartitionCols:_col0, _col1, _col2
Group By Operator [GBY_45] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count()"],keys:_col4, _col5, _col6
- Merge Join Operator [MERGEJOIN_1135] (rows=7620440 width=11)
- Conds:RS_41._col1=RS_1336._col0(Inner),Output:["_col4","_col5","_col6"]
+ Merge Join Operator [MERGEJOIN_1195] (rows=7620440 width=11)
+ Conds:RS_41._col1=RS_1396._col0(Inner),Output:["_col4","_col5","_col6"]
<-Map 62 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1336]
+ SHUFFLE [RS_1396]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1325]
+ Please refer to the previous Select Operator [SEL_1385]
<-Reducer 39 [SIMPLE_EDGE]
SHUFFLE [RS_41]
PartitionCols:_col1
- Please refer to the previous Merge Join Operator [MERGEJOIN_1134]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_1194]
<-Reducer 48 [CONTAINS] vectorized
- Reduce Output Operator [RS_1430]
+ Reduce Output Operator [RS_1490]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_1429] (rows=121728 width=19)
+ Group By Operator [GBY_1489] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(_col3)"],keys:_col0, _col1, _col2
- Group By Operator [GBY_1428] (rows=121728 width=19)
+ Group By Operator [GBY_1488] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 47 [SIMPLE_EDGE]
SHUFFLE [RS_67]
PartitionCols:_col0, _col1, _col2
Group By Operator [GBY_66] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count()"],keys:_col4, _col5, _col6
- Merge Join Operator [MERGEJOIN_1137] (rows=3828623 width=11)
- Conds:RS_62._col1=RS_1337._col0(Inner),Output:["_col4","_col5","_col6"]
+ Merge Join Operator [MERGEJOIN_1197] (rows=3828623 width=11)
+ Conds:RS_62._col1=RS_1397._col0(Inner),Output:["_col4","_col5","_col6"]
<-Map 62 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1337]
+ SHUFFLE [RS_1397]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1325]
+ Please refer to the previous Select Operator [SEL_1385]
<-Reducer 46 [SIMPLE_EDGE]
SHUFFLE [RS_62]
PartitionCols:_col1
- Please refer to the previous Merge Join Operator [MERGEJOIN_1136]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_1196]
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query17.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query17.q.out
index 8781989..eb04b5e 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query17.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query17.q.out
@@ -122,16 +122,16 @@ Stage-0
limit:100
Stage-1
Reducer 7 vectorized
- File Output Operator [FS_253]
- Limit [LIM_252] (rows=100 width=466)
+ File Output Operator [FS_288]
+ Limit [LIM_287] (rows=100 width=466)
Number of rows:100
- Select Operator [SEL_251] (rows=8581091679 width=466)
+ Select Operator [SEL_286] (rows=8581091679 width=466)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"]
<-Reducer 6 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_250]
- Select Operator [SEL_249] (rows=8581091679 width=466)
+ SHUFFLE [RS_285]
+ Select Operator [SEL_284] (rows=8581091679 width=466)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- Group By Operator [GBY_248] (rows=8581091679 width=466)
+ Group By Operator [GBY_283] (rows=8581091679 width=466)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"],aggregations:["count(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)","count(VALUE._col4)","sum(VALUE._col5)","sum(VALUE._col6)","sum(VALUE._col7)","count(VALUE._col8)","sum(VALUE._col9)","sum(VALUE._col10)","sum(VALUE._col11)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_48]
@@ -142,124 +142,124 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
Top N Key Operator [TNK_97] (rows=8581091679 width=381)
keys:_col21, _col22, _col19,top n:100
- Merge Join Operator [MERGEJOIN_215] (rows=8581091679 width=381)
- Conds:RS_42._col6=RS_247._col0(Inner),Output:["_col3","_col10","_col16","_col19","_col21","_col22"]
+ Merge Join Operator [MERGEJOIN_250] (rows=8581091679 width=381)
+ Conds:RS_42._col6=RS_282._col0(Inner),Output:["_col3","_col10","_col16","_col19","_col21","_col22"]
<-Map 17 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_247]
+ SHUFFLE [RS_282]
PartitionCols:_col0
- Select Operator [SEL_246] (rows=462000 width=288)
+ Select Operator [SEL_281] (rows=462000 width=288)
Output:["_col0","_col1","_col2"]
TableScan [TS_31] (rows=462000 width=288)
default@item,item,Tbl:COMPLETE,Col:COMPLETE,Output:["i_item_sk","i_item_id","i_item_desc"]
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_42]
PartitionCols:_col6
- Merge Join Operator [MERGEJOIN_214] (rows=8581091679 width=101)
- Conds:RS_39._col8=RS_245._col0(Inner),Output:["_col3","_col6","_col10","_col16","_col19"]
+ Merge Join Operator [MERGEJOIN_249] (rows=8581091679 width=101)
+ Conds:RS_39._col8=RS_280._col0(Inner),Output:["_col3","_col6","_col10","_col16","_col19"]
<-Map 16 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_245]
+ SHUFFLE [RS_280]
PartitionCols:_col0
- Select Operator [SEL_244] (rows=1704 width=90)
+ Select Operator [SEL_279] (rows=1704 width=90)
Output:["_col0","_col1"]
TableScan [TS_29] (rows=1704 width=90)
default@store,store,Tbl:COMPLETE,Col:COMPLETE,Output:["s_store_sk","s_state"]
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_39]
PartitionCols:_col8
- Merge Join Operator [MERGEJOIN_213] (rows=8581091679 width=19)
+ Merge Join Operator [MERGEJOIN_248] (rows=8581091679 width=19)
Conds:RS_36._col1, _col2=RS_37._col9, _col8(Inner),Output:["_col3","_col6","_col8","_col10","_col16"]
<-Reducer 11 [SIMPLE_EDGE]
SHUFFLE [RS_37]
PartitionCols:_col9, _col8
- Merge Join Operator [MERGEJOIN_212] (rows=42605597 width=19)
+ Merge Join Operator [MERGEJOIN_247] (rows=42605597 width=19)
Conds:RS_25._col2, _col1, _col4=RS_26._col2, _col1, _col3(Inner),Output:["_col1","_col3","_col5","_col8","_col9","_col11"]
<-Reducer 10 [SIMPLE_EDGE]
SHUFFLE [RS_25]
PartitionCols:_col2, _col1, _col4
- Merge Join Operator [MERGEJOIN_210] (rows=27749405 width=10)
- Conds:RS_240._col0=RS_224._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5"]
+ Merge Join Operator [MERGEJOIN_245] (rows=27749405 width=10)
+ Conds:RS_275._col0=RS_259._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5"]
<-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_224]
+ PARTITION_ONLY_SHUFFLE [RS_259]
PartitionCols:_col0
- Select Operator [SEL_220] (rows=101 width=4)
+ Select Operator [SEL_255] (rows=101 width=4)
Output:["_col0"]
- Filter Operator [FIL_217] (rows=101 width=94)
+ Filter Operator [FIL_252] (rows=101 width=94)
predicate:(d_quarter_name = '2000Q1')
TableScan [TS_3] (rows=73049 width=94)
default@date_dim,d3,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_quarter_name"]
<-Map 14 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_240]
+ SHUFFLE [RS_275]
PartitionCols:_col0
- Select Operator [SEL_239] (rows=501694138 width=23)
+ Select Operator [SEL_274] (rows=501694138 width=23)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_238] (rows=501694138 width=23)
+ Filter Operator [FIL_273] (rows=501694138 width=23)
predicate:(ss_sold_date_sk is not null and ss_customer_sk is not null and ss_store_sk is not null and ss_sold_date_sk BETWEEN DynamicValue(RS_23_d1_d_date_sk_min) AND DynamicValue(RS_23_d1_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_23_d1_d_date_sk_bloom_filter)))
TableScan [TS_6] (rows=575995635 width=23)
default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_item_sk","ss_customer_sk","ss_store_sk","ss_ticket_number","ss_quantity"]
<-Reducer 12 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_237]
- Group By Operator [GBY_236] (rows=1 width=12)
+ BROADCAST [RS_272]
+ Group By Operator [GBY_271] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_230]
- Group By Operator [GBY_228] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_265]
+ Group By Operator [GBY_263] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_225] (rows=101 width=4)
+ Select Operator [SEL_260] (rows=101 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_220]
+ Please refer to the previous Select Operator [SEL_255]
<-Reducer 13 [SIMPLE_EDGE]
SHUFFLE [RS_26]
PartitionCols:_col2, _col1, _col3
- Merge Join Operator [MERGEJOIN_211] (rows=8143830 width=14)
- Conds:RS_243._col0=RS_226._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_246] (rows=8143830 width=14)
+ Conds:RS_278._col0=RS_261._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
<-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_226]
+ PARTITION_ONLY_SHUFFLE [RS_261]
PartitionCols:_col0
- Select Operator [SEL_221] (rows=304 width=4)
+ Select Operator [SEL_256] (rows=304 width=4)
Output:["_col0"]
- Filter Operator [FIL_218] (rows=304 width=94)
+ Filter Operator [FIL_253] (rows=304 width=94)
predicate:(d_quarter_name) IN ('2000Q1', '2000Q2', '2000Q3')
Please refer to the previous TableScan [TS_3]
<-Map 15 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_243]
+ SHUFFLE [RS_278]
PartitionCols:_col0
- Select Operator [SEL_242] (rows=53632139 width=19)
+ Select Operator [SEL_277] (rows=53632139 width=19)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_241] (rows=53632139 width=19)
+ Filter Operator [FIL_276] (rows=53632139 width=19)
predicate:(sr_customer_sk is not null and sr_returned_date_sk is not null)
TableScan [TS_12] (rows=57591150 width=19)
default@store_returns,store_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["sr_returned_date_sk","sr_item_sk","sr_customer_sk","sr_ticket_number","sr_return_quantity"]
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_36]
PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_209] (rows=47131396 width=11)
- Conds:RS_235._col0=RS_222._col0(Inner),Output:["_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_244] (rows=47131396 width=11)
+ Conds:RS_270._col0=RS_257._col0(Inner),Output:["_col1","_col2","_col3"]
<-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_222]
+ PARTITION_ONLY_SHUFFLE [RS_257]
PartitionCols:_col0
- Select Operator [SEL_219] (rows=304 width=4)
+ Select Operator [SEL_254] (rows=304 width=4)
Output:["_col0"]
- Filter Operator [FIL_216] (rows=304 width=94)
+ Filter Operator [FIL_251] (rows=304 width=94)
predicate:(d_quarter_name) IN ('2000Q1', '2000Q2', '2000Q3')
Please refer to the previous TableScan [TS_3]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_235]
+ SHUFFLE [RS_270]
PartitionCols:_col0
- Select Operator [SEL_234] (rows=285117831 width=15)
+ Select Operator [SEL_269] (rows=285117831 width=15)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_233] (rows=285117831 width=15)
+ Filter Operator [FIL_268] (rows=285117831 width=15)
predicate:(cs_sold_date_sk is not null and cs_bill_customer_sk is not null and cs_sold_date_sk BETWEEN DynamicValue(RS_34_d3_d_date_sk_min) AND DynamicValue(RS_34_d3_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_34_d3_d_date_sk_bloom_filter)))
TableScan [TS_0] (rows=287989836 width=15)
default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["cs_sold_date_sk","cs_bill_customer_sk","cs_item_sk","cs_quantity"]
<-Reducer 9 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_232]
- Group By Operator [GBY_231] (rows=1 width=12)
+ BROADCAST [RS_267]
+ Group By Operator [GBY_266] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_229]
- Group By Operator [GBY_227] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_264]
+ Group By Operator [GBY_262] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_223] (rows=304 width=4)
+ Select Operator [SEL_258] (rows=304 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_219]
+ Please refer to the previous Select Operator [SEL_254]
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query24.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query24.q.out
index 87895bd..6528f87 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query24.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query24.q.out
@@ -1,4 +1,4 @@
-Warning: Shuffle Join MERGEJOIN[301][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 6' is a cross product
+Warning: Shuffle Join MERGEJOIN[331][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 6' is a cross product
PREHOOK: query: explain
with ssales as
(select c_last_name
@@ -116,22 +116,18 @@ POSTHOOK: Output: hdfs://### HDFS PATH ###
Plan optimized by CBO.
Vertex dependency in root stage
-Map 1 <- Reducer 20 (BROADCAST_EDGE)
-Map 21 <- Reducer 16 (BROADCAST_EDGE)
-Reducer 10 <- Map 19 (SIMPLE_EDGE), Reducer 9 (SIMPLE_EDGE)
-Reducer 11 <- Reducer 10 (SIMPLE_EDGE)
-Reducer 12 <- Reducer 11 (CUSTOM_SIMPLE_EDGE)
-Reducer 14 <- Map 13 (SIMPLE_EDGE), Map 17 (SIMPLE_EDGE)
-Reducer 15 <- Map 18 (SIMPLE_EDGE), Reducer 14 (SIMPLE_EDGE)
-Reducer 16 <- Reducer 15 (CUSTOM_SIMPLE_EDGE)
-Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 7 (SIMPLE_EDGE)
-Reducer 20 <- Map 19 (CUSTOM_SIMPLE_EDGE)
-Reducer 3 <- Reducer 15 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
-Reducer 4 <- Map 19 (SIMPLE_EDGE), Reducer 3 (SIMPLE_EDGE)
+Map 1 <- Reducer 14 (BROADCAST_EDGE)
+Reducer 12 <- Map 11 (SIMPLE_EDGE), Map 15 (SIMPLE_EDGE)
+Reducer 13 <- Map 16 (SIMPLE_EDGE), Reducer 12 (SIMPLE_EDGE)
+Reducer 14 <- Reducer 13 (CUSTOM_SIMPLE_EDGE)
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 10 (SIMPLE_EDGE)
+Reducer 3 <- Reducer 13 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
+Reducer 4 <- Map 17 (SIMPLE_EDGE), Reducer 3 (SIMPLE_EDGE)
Reducer 5 <- Reducer 4 (SIMPLE_EDGE)
-Reducer 6 <- Reducer 12 (CUSTOM_SIMPLE_EDGE), Reducer 5 (CUSTOM_SIMPLE_EDGE)
-Reducer 8 <- Map 21 (SIMPLE_EDGE), Map 7 (SIMPLE_EDGE)
-Reducer 9 <- Reducer 15 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
+Reducer 6 <- Reducer 5 (CUSTOM_SIMPLE_EDGE), Reducer 9 (CUSTOM_SIMPLE_EDGE)
+Reducer 7 <- Map 17 (SIMPLE_EDGE), Reducer 3 (SIMPLE_EDGE)
+Reducer 8 <- Reducer 7 (SIMPLE_EDGE)
+Reducer 9 <- Reducer 8 (CUSTOM_SIMPLE_EDGE)
Stage-0
Fetch Operator
@@ -143,179 +139,145 @@ Stage-0
Output:["_col0","_col1","_col2","_col3"]
Filter Operator [FIL_94] (rows=5130 width=492)
predicate:(_col3 > _col4)
- Merge Join Operator [MERGEJOIN_301] (rows=15392 width=492)
+ Merge Join Operator [MERGEJOIN_331] (rows=15392 width=492)
Conds:(Inner),Output:["_col0","_col1","_col2","_col3","_col4"]
- <-Reducer 12 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_345]
- Select Operator [SEL_344] (rows=1 width=112)
- Output:["_col0"]
- Filter Operator [FIL_343] (rows=1 width=120)
- predicate:CAST( (_col0 / _col1) AS decimal(21,6)) is not null
- Group By Operator [GBY_342] (rows=1 width=120)
- Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"]
- <-Reducer 11 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_341]
- Group By Operator [GBY_340] (rows=1 width=120)
- Output:["_col0","_col1"],aggregations:["sum(_col10)","count(_col10)"]
- Select Operator [SEL_339] (rows=589731268 width=932)
- Output:["_col10"]
- Group By Operator [GBY_338] (rows=589731268 width=932)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5, KEY._col6, KEY._col7, KEY._col8, KEY._col9
- <-Reducer 10 [SIMPLE_EDGE]
- SHUFFLE [RS_81]
- PartitionCols:_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
- Group By Operator [GBY_80] (rows=589731268 width=932)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10"],aggregations:["sum(_col4)"],keys:_col9, _col10, _col13, _col17, _col18, _col21, _col22, _col23, _col24, _col25
- Merge Join Operator [MERGEJOIN_300] (rows=589731268 width=928)
- Conds:RS_76._col0=RS_305._col0(Inner),Output:["_col4","_col9","_col10","_col13","_col17","_col18","_col21","_col22","_col23","_col24","_col25"]
- <-Map 19 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_305]
- PartitionCols:_col0
- Select Operator [SEL_303] (rows=462000 width=384)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_22] (rows=462000 width=384)
- default@item,item,Tbl:COMPLETE,Col:COMPLETE,Output:["i_item_sk","i_current_price","i_size","i_color","i_units","i_manager_id"]
- <-Reducer 9 [SIMPLE_EDGE]
- SHUFFLE [RS_76]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_299] (rows=589731268 width=551)
- Conds:RS_73._col2, _col1=RS_74._col9, _col0(Inner),Output:["_col0","_col4","_col9","_col10","_col13","_col17","_col18"]
- <-Reducer 15 [SIMPLE_EDGE]
- SHUFFLE [RS_74]
- PartitionCols:_col9, _col0
- Select Operator [SEL_21] (rows=7276996 width=724)
- Output:["_col0","_col2","_col3","_col6","_col9","_col10","_col11"]
- Filter Operator [FIL_20] (rows=7276996 width=724)
- predicate:(_col12 <> _col3)
- Merge Join Operator [MERGEJOIN_293] (rows=7276996 width=724)
- Conds:RS_17._col0=RS_326._col1(Inner),Output:["_col1","_col3","_col4","_col5","_col6","_col8","_col10","_col11","_col12"]
- <-Map 18 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_326]
- PartitionCols:_col1
- Select Operator [SEL_325] (rows=80000000 width=280)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_324] (rows=80000000 width=280)
- predicate:c_current_addr_sk is not null
- TableScan [TS_11] (rows=80000000 width=280)
- default@customer,customer,Tbl:COMPLETE,Col:COMPLETE,Output:["c_customer_sk","c_current_addr_sk","c_first_name","c_last_name","c_birth_country"]
- <-Reducer 14 [SIMPLE_EDGE]
- SHUFFLE [RS_17]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_292] (rows=611379 width=452)
- Conds:RS_320._col2=RS_323._col3(Inner),Output:["_col0","_col1","_col3","_col4","_col5","_col6"]
- <-Map 13 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_320]
- PartitionCols:_col2
- Select Operator [SEL_319] (rows=40000000 width=363)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_318] (rows=40000000 width=276)
- predicate:ca_zip is not null
- TableScan [TS_5] (rows=40000000 width=276)
- default@customer_address,customer_address,Tbl:COMPLETE,Col:COMPLETE,Output:["ca_address_sk","ca_state","ca_zip","ca_country"]
- <-Map 17 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_323]
- PartitionCols:_col3
- Select Operator [SEL_322] (rows=155 width=267)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_321] (rows=155 width=271)
- predicate:((s_market_id = 7) and s_zip is not null)
- TableScan [TS_8] (rows=1704 width=270)
- default@store,store,Tbl:COMPLETE,Col:COMPLETE,Output:["s_store_sk","s_store_name","s_market_id","s_state","s_zip"]
- <-Reducer 8 [SIMPLE_EDGE]
- SHUFFLE [RS_73]
- PartitionCols:_col2, _col1
- Merge Join Operator [MERGEJOIN_296] (rows=537799798 width=118)
- Conds:RS_337._col0, _col3=RS_317._col0, _col1(Inner),Output:["_col0","_col1","_col2","_col4"]
- <-Map 7 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_317]
- PartitionCols:_col0, _col1
- Select Operator [SEL_315] (rows=57591150 width=8)
- Output:["_col0","_col1"]
- TableScan [TS_3] (rows=57591150 width=8)
- default@store_returns,store_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["sr_item_sk","sr_ticket_number"]
- <-Map 21 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_337]
- PartitionCols:_col0, _col3
- Select Operator [SEL_336] (rows=525333486 width=122)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_335] (rows=525333486 width=122)
- predicate:(ss_customer_sk is not null and ss_store_sk is not null and ss_customer_sk BETWEEN DynamicValue(RS_74_customer_c_customer_sk_min) AND DynamicValue(RS_74_customer_c_customer_sk_max) and in_bloom_filter(ss_customer_sk, DynamicValue(RS_74_customer_c_customer_sk_bloom_filter)))
- TableScan [TS_46] (rows=575995635 width=122)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_item_sk","ss_customer_sk","ss_store_sk","ss_ticket_number","ss_sales_price"]
- <-Reducer 16 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_334]
- Group By Operator [GBY_333] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=6636187)"]
- <-Reducer 15 [CUSTOM_SIMPLE_EDGE]
- SHUFFLE [RS_235]
- Group By Operator [GBY_234] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=6636187)"]
- Select Operator [SEL_233] (rows=7276996 width=4)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_21]
<-Reducer 5 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_332]
- Filter Operator [FIL_331] (rows=15392 width=380)
+ PARTITION_ONLY_SHUFFLE [RS_358]
+ Filter Operator [FIL_357] (rows=15392 width=380)
predicate:_col3 is not null
- Select Operator [SEL_330] (rows=15392 width=380)
+ Select Operator [SEL_356] (rows=15392 width=380)
Output:["_col0","_col1","_col2","_col3"]
- Group By Operator [GBY_329] (rows=15392 width=380)
+ Group By Operator [GBY_355] (rows=15392 width=380)
Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(_col9)"],keys:_col0, _col1, _col3
- Select Operator [SEL_328] (rows=86004082 width=843)
+ Select Operator [SEL_354] (rows=86004082 width=843)
Output:["_col0","_col1","_col3","_col9"]
- Group By Operator [GBY_327] (rows=86004082 width=843)
+ Group By Operator [GBY_353] (rows=86004082 width=843)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5, KEY._col6, KEY._col7, KEY._col8
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_36]
PartitionCols:_col0, _col1, _col2
Group By Operator [GBY_35] (rows=86004082 width=843)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"],aggregations:["sum(_col4)"],keys:_col9, _col10, _col17, _col13, _col18, _col21, _col22, _col23, _col24
- Merge Join Operator [MERGEJOIN_295] (rows=86004082 width=813)
- Conds:RS_31._col0=RS_306._col0(Inner),Output:["_col4","_col9","_col10","_col13","_col17","_col18","_col21","_col22","_col23","_col24"]
- <-Map 19 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_306]
+ Merge Join Operator [MERGEJOIN_325] (rows=86004082 width=813)
+ Conds:RS_31._col0=RS_352._col0(Inner),Output:["_col4","_col9","_col10","_col13","_col17","_col18","_col21","_col22","_col23","_col24"]
+ <-Map 17 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_352]
PartitionCols:_col0
- Select Operator [SEL_304] (rows=7000 width=295)
+ Select Operator [SEL_350] (rows=7000 width=295)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_302] (rows=7000 width=384)
+ Filter Operator [FIL_348] (rows=7000 width=384)
predicate:(i_color = 'orchid')
- Please refer to the previous TableScan [TS_22]
+ TableScan [TS_22] (rows=462000 width=384)
+ default@item,item,Tbl:COMPLETE,Col:COMPLETE,Output:["i_item_sk","i_current_price","i_size","i_color","i_units","i_manager_id"]
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_31]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_294] (rows=589731268 width=551)
+ Merge Join Operator [MERGEJOIN_324] (rows=589731268 width=551)
Conds:RS_28._col2, _col1=RS_29._col9, _col0(Inner),Output:["_col0","_col4","_col9","_col10","_col13","_col17","_col18"]
- <-Reducer 15 [SIMPLE_EDGE]
+ <-Reducer 13 [SIMPLE_EDGE]
SHUFFLE [RS_29]
PartitionCols:_col9, _col0
- Please refer to the previous Select Operator [SEL_21]
+ Select Operator [SEL_21] (rows=7276996 width=724)
+ Output:["_col0","_col2","_col3","_col6","_col9","_col10","_col11"]
+ Filter Operator [FIL_20] (rows=7276996 width=724)
+ predicate:(_col12 <> _col3)
+ Merge Join Operator [MERGEJOIN_323] (rows=7276996 width=724)
+ Conds:RS_17._col0=RS_340._col1(Inner),Output:["_col1","_col3","_col4","_col5","_col6","_col8","_col10","_col11","_col12"]
+ <-Map 16 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_340]
+ PartitionCols:_col1
+ Select Operator [SEL_339] (rows=80000000 width=280)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_338] (rows=80000000 width=280)
+ predicate:c_current_addr_sk is not null
+ TableScan [TS_11] (rows=80000000 width=280)
+ default@customer,customer,Tbl:COMPLETE,Col:COMPLETE,Output:["c_customer_sk","c_current_addr_sk","c_first_name","c_last_name","c_birth_country"]
+ <-Reducer 12 [SIMPLE_EDGE]
+ SHUFFLE [RS_17]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_322] (rows=611379 width=452)
+ Conds:RS_334._col2=RS_337._col3(Inner),Output:["_col0","_col1","_col3","_col4","_col5","_col6"]
+ <-Map 11 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_334]
+ PartitionCols:_col2
+ Select Operator [SEL_333] (rows=40000000 width=363)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_332] (rows=40000000 width=276)
+ predicate:ca_zip is not null
+ TableScan [TS_5] (rows=40000000 width=276)
+ default@customer_address,customer_address,Tbl:COMPLETE,Col:COMPLETE,Output:["ca_address_sk","ca_state","ca_zip","ca_country"]
+ <-Map 15 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_337]
+ PartitionCols:_col3
+ Select Operator [SEL_336] (rows=155 width=267)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_335] (rows=155 width=271)
+ predicate:((s_market_id = 7) and s_zip is not null)
+ TableScan [TS_8] (rows=1704 width=270)
+ default@store,store,Tbl:COMPLETE,Col:COMPLETE,Output:["s_store_sk","s_store_name","s_market_id","s_state","s_zip"]
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_28]
PartitionCols:_col2, _col1
- Merge Join Operator [MERGEJOIN_291] (rows=537799798 width=118)
- Conds:RS_314._col0, _col3=RS_316._col0, _col1(Inner),Output:["_col0","_col1","_col2","_col4"]
- <-Map 7 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_316]
- PartitionCols:_col0, _col1
- Please refer to the previous Select Operator [SEL_315]
+ Merge Join Operator [MERGEJOIN_321] (rows=537799798 width=118)
+ Conds:RS_345._col0, _col3=RS_347._col0, _col1(Inner),Output:["_col0","_col1","_col2","_col4"]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_314]
+ SHUFFLE [RS_345]
PartitionCols:_col0, _col3
- Select Operator [SEL_313] (rows=525333486 width=122)
+ Select Operator [SEL_344] (rows=525333486 width=122)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_312] (rows=525333486 width=122)
- predicate:(ss_customer_sk is not null and ss_store_sk is not null and ss_item_sk BETWEEN DynamicValue(RS_32_item_i_item_sk_min) AND DynamicValue(RS_32_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_32_item_i_item_sk_bloom_filter)))
+ Filter Operator [FIL_343] (rows=525333486 width=122)
+ predicate:(ss_customer_sk is not null and ss_store_sk is not null and ss_store_sk BETWEEN DynamicValue(RS[300]_col0) AND DynamicValue(RS[300]_col1) and ss_customer_sk BETWEEN DynamicValue(RS[300]_col2) AND DynamicValue(RS[300]_col3) and in_bloom_filter(hash(ss_store_sk,ss_customer_sk), DynamicValue(RS[300]_col4)))
TableScan [TS_0] (rows=575995635 width=122)
default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_item_sk","ss_customer_sk","ss_store_sk","ss_ticket_number","ss_sales_price"]
- <-Reducer 20 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_311]
- Group By Operator [GBY_310] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 19 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_309]
- Group By Operator [GBY_308] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_307] (rows=7000 width=4)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_304]
+ <-Reducer 14 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_342]
+ Group By Operator [GBY_341] (rows=1 width=160)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","min(VALUE._col2)","max(VALUE._col3)","bloom_filter(VALUE._col4, expectedEntries=7276996)"]
+ <-Reducer 13 [CUSTOM_SIMPLE_EDGE]
+ SHUFFLE [RS_298]
+ Group By Operator [GBY_297] (rows=1 width=160)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["min(_col0)","max(_col0)","min(_col1)","max(_col1)","bloom_filter(_col3, expectedEntries=7276996)"]
+ Select Operator [SEL_296] (rows=7276996 width=12)
+ Output:["_col0","_col1","_col3"]
+ Please refer to the previous Select Operator [SEL_21]
+ <-Map 10 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_347]
+ PartitionCols:_col0, _col1
+ Select Operator [SEL_346] (rows=57591150 width=8)
+ Output:["_col0","_col1"]
+ TableScan [TS_3] (rows=57591150 width=8)
+ default@store_returns,store_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["sr_item_sk","sr_ticket_number"]
+ <-Reducer 9 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_366]
+ Select Operator [SEL_365] (rows=1 width=112)
+ Output:["_col0"]
+ Filter Operator [FIL_364] (rows=1 width=120)
+ predicate:CAST( (_col0 / _col1) AS decimal(21,6)) is not null
+ Group By Operator [GBY_363] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"]
+ <-Reducer 8 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_362]
+ Group By Operator [GBY_361] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(_col10)","count(_col10)"]
+ Select Operator [SEL_360] (rows=589731268 width=932)
+ Output:["_col10"]
+ Group By Operator [GBY_359] (rows=589731268 width=932)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5, KEY._col6, KEY._col7, KEY._col8, KEY._col9
+ <-Reducer 7 [SIMPLE_EDGE]
+ SHUFFLE [RS_81]
+ PartitionCols:_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
+ Group By Operator [GBY_80] (rows=589731268 width=932)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10"],aggregations:["sum(_col4)"],keys:_col9, _col10, _col13, _col17, _col18, _col21, _col22, _col23, _col24, _col25
+ Merge Join Operator [MERGEJOIN_330] (rows=589731268 width=928)
+ Conds:RS_76._col0=RS_351._col0(Inner),Output:["_col4","_col9","_col10","_col13","_col17","_col18","_col21","_col22","_col23","_col24","_col25"]
+ <-Map 17 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_351]
+ PartitionCols:_col0
+ Select Operator [SEL_349] (rows=462000 width=384)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Please refer to the previous TableScan [TS_22]
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_76]
+ PartitionCols:_col0
+ Please refer to the previous Merge Join Operator [MERGEJOIN_324]
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query25.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query25.q.out
index 22c03eb..97e5f23 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query25.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query25.q.out
@@ -128,14 +128,14 @@ Stage-0
limit:100
Stage-1
Reducer 7 vectorized
- File Output Operator [FS_251]
- Limit [LIM_250] (rows=100 width=808)
+ File Output Operator [FS_286]
+ Limit [LIM_285] (rows=100 width=808)
Number of rows:100
- Select Operator [SEL_249] (rows=4248052730 width=808)
+ Select Operator [SEL_284] (rows=4248052730 width=808)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
<-Reducer 6 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_248]
- Group By Operator [GBY_247] (rows=4248052730 width=808)
+ SHUFFLE [RS_283]
+ Group By Operator [GBY_282] (rows=4248052730 width=808)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_47]
@@ -144,124 +144,124 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["sum(_col10)","sum(_col16)","sum(_col3)"],keys:_col22, _col23, _col19, _col20
Top N Key Operator [TNK_96] (rows=4248052730 width=807)
keys:_col22, _col23, _col19, _col20,top n:100
- Merge Join Operator [MERGEJOIN_214] (rows=4248052730 width=807)
- Conds:RS_42._col6=RS_246._col0(Inner),Output:["_col3","_col10","_col16","_col19","_col20","_col22","_col23"]
+ Merge Join Operator [MERGEJOIN_249] (rows=4248052730 width=807)
+ Conds:RS_42._col6=RS_281._col0(Inner),Output:["_col3","_col10","_col16","_col19","_col20","_col22","_col23"]
<-Map 17 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_246]
+ SHUFFLE [RS_281]
PartitionCols:_col0
- Select Operator [SEL_245] (rows=462000 width=288)
+ Select Operator [SEL_280] (rows=462000 width=288)
Output:["_col0","_col1","_col2"]
TableScan [TS_31] (rows=462000 width=288)
default@item,item,Tbl:COMPLETE,Col:COMPLETE,Output:["i_item_sk","i_item_id","i_item_desc"]
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_42]
PartitionCols:_col6
- Merge Join Operator [MERGEJOIN_213] (rows=4248052730 width=527)
- Conds:RS_39._col8=RS_244._col0(Inner),Output:["_col3","_col6","_col10","_col16","_col19","_col20"]
+ Merge Join Operator [MERGEJOIN_248] (rows=4248052730 width=527)
+ Conds:RS_39._col8=RS_279._col0(Inner),Output:["_col3","_col6","_col10","_col16","_col19","_col20"]
<-Map 16 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_244]
+ SHUFFLE [RS_279]
PartitionCols:_col0
- Select Operator [SEL_243] (rows=1704 width=192)
+ Select Operator [SEL_278] (rows=1704 width=192)
Output:["_col0","_col1","_col2"]
TableScan [TS_29] (rows=1704 width=192)
default@store,store,Tbl:COMPLETE,Col:COMPLETE,Output:["s_store_sk","s_store_id","s_store_name"]
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_39]
PartitionCols:_col8
- Merge Join Operator [MERGEJOIN_212] (rows=4248052730 width=343)
+ Merge Join Operator [MERGEJOIN_247] (rows=4248052730 width=343)
Conds:RS_36._col1, _col2=RS_37._col9, _col8(Inner),Output:["_col3","_col6","_col8","_col10","_col16"]
<-Reducer 11 [SIMPLE_EDGE]
SHUFFLE [RS_37]
PartitionCols:_col9, _col8
- Merge Join Operator [MERGEJOIN_211] (rows=21091882 width=154)
+ Merge Join Operator [MERGEJOIN_246] (rows=21091882 width=154)
Conds:RS_25._col2, _col1, _col4=RS_26._col2, _col1, _col3(Inner),Output:["_col1","_col3","_col5","_col8","_col9","_col11"]
<-Reducer 10 [SIMPLE_EDGE]
SHUFFLE [RS_25]
PartitionCols:_col2, _col1, _col4
- Merge Join Operator [MERGEJOIN_209] (rows=13737330 width=8)
- Conds:RS_239._col0=RS_223._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5"]
+ Merge Join Operator [MERGEJOIN_244] (rows=13737330 width=8)
+ Conds:RS_274._col0=RS_258._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5"]
<-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_223]
+ PARTITION_ONLY_SHUFFLE [RS_258]
PartitionCols:_col0
- Select Operator [SEL_219] (rows=50 width=4)
+ Select Operator [SEL_254] (rows=50 width=4)
Output:["_col0"]
- Filter Operator [FIL_216] (rows=50 width=12)
+ Filter Operator [FIL_251] (rows=50 width=12)
predicate:((d_year = 2000) and (d_moy = 4))
TableScan [TS_3] (rows=73049 width=12)
default@date_dim,d3,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_year","d_moy"]
<-Map 14 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_239]
+ SHUFFLE [RS_274]
PartitionCols:_col0
- Select Operator [SEL_238] (rows=501694138 width=126)
+ Select Operator [SEL_273] (rows=501694138 width=126)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_237] (rows=501694138 width=126)
+ Filter Operator [FIL_272] (rows=501694138 width=126)
predicate:(ss_sold_date_sk is not null and ss_customer_sk is not null and ss_store_sk is not null and ss_sold_date_sk BETWEEN DynamicValue(RS_23_d1_d_date_sk_min) AND DynamicValue(RS_23_d1_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_23_d1_d_date_sk_bloom_filter)))
TableScan [TS_6] (rows=575995635 width=126)
default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_item_sk","ss_customer_sk","ss_store_sk","ss_ticket_number","ss_net_profit"]
<-Reducer 12 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_236]
- Group By Operator [GBY_235] (rows=1 width=12)
+ BROADCAST [RS_271]
+ Group By Operator [GBY_270] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_229]
- Group By Operator [GBY_227] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_264]
+ Group By Operator [GBY_262] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_224] (rows=50 width=4)
+ Select Operator [SEL_259] (rows=50 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_219]
+ Please refer to the previous Select Operator [SEL_254]
<-Reducer 13 [SIMPLE_EDGE]
SHUFFLE [RS_26]
PartitionCols:_col2, _col1, _col3
- Merge Join Operator [MERGEJOIN_210] (rows=9402909 width=100)
- Conds:RS_242._col0=RS_225._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_245] (rows=9402909 width=100)
+ Conds:RS_277._col0=RS_260._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
<-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_225]
+ PARTITION_ONLY_SHUFFLE [RS_260]
PartitionCols:_col0
- Select Operator [SEL_220] (rows=351 width=4)
+ Select Operator [SEL_255] (rows=351 width=4)
Output:["_col0"]
- Filter Operator [FIL_217] (rows=351 width=12)
+ Filter Operator [FIL_252] (rows=351 width=12)
predicate:((d_year = 2000) and d_moy BETWEEN 4 AND 10)
Please refer to the previous TableScan [TS_3]
<-Map 15 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_242]
+ SHUFFLE [RS_277]
PartitionCols:_col0
- Select Operator [SEL_241] (rows=53632139 width=123)
+ Select Operator [SEL_276] (rows=53632139 width=123)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_240] (rows=53632139 width=123)
+ Filter Operator [FIL_275] (rows=53632139 width=123)
predicate:(sr_customer_sk is not null and sr_returned_date_sk is not null)
TableScan [TS_12] (rows=57591150 width=123)
default@store_returns,store_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["sr_returned_date_sk","sr_item_sk","sr_customer_sk","sr_ticket_number","sr_net_loss"]
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_36]
PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_208] (rows=54418158 width=119)
- Conds:RS_234._col0=RS_221._col0(Inner),Output:["_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_243] (rows=54418158 width=119)
+ Conds:RS_269._col0=RS_256._col0(Inner),Output:["_col1","_col2","_col3"]
<-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_221]
+ PARTITION_ONLY_SHUFFLE [RS_256]
PartitionCols:_col0
- Select Operator [SEL_218] (rows=351 width=4)
+ Select Operator [SEL_253] (rows=351 width=4)
Output:["_col0"]
- Filter Operator [FIL_215] (rows=351 width=12)
+ Filter Operator [FIL_250] (rows=351 width=12)
predicate:((d_year = 2000) and d_moy BETWEEN 4 AND 10)
Please refer to the previous TableScan [TS_3]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_234]
+ SHUFFLE [RS_269]
PartitionCols:_col0
- Select Operator [SEL_233] (rows=285117831 width=123)
+ Select Operator [SEL_268] (rows=285117831 width=123)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_232] (rows=285117831 width=123)
+ Filter Operator [FIL_267] (rows=285117831 width=123)
predicate:(cs_sold_date_sk is not null and cs_bill_customer_sk is not null and cs_sold_date_sk BETWEEN DynamicValue(RS_34_d3_d_date_sk_min) AND DynamicValue(RS_34_d3_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_34_d3_d_date_sk_bloom_filter)))
TableScan [TS_0] (rows=287989836 width=123)
default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["cs_sold_date_sk","cs_bill_customer_sk","cs_item_sk","cs_net_profit"]
<-Reducer 9 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_231]
- Group By Operator [GBY_230] (rows=1 width=12)
+ BROADCAST [RS_266]
+ Group By Operator [GBY_265] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_228]
- Group By Operator [GBY_226] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_263]
+ Group By Operator [GBY_261] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_222] (rows=351 width=4)
+ Select Operator [SEL_257] (rows=351 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_218]
+ Please refer to the previous Select Operator [SEL_253]
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query29.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query29.q.out
index 1de67a7..0cfad85 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query29.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query29.q.out
@@ -126,14 +126,14 @@ Stage-0
limit:100
Stage-1
Reducer 7 vectorized
- File Output Operator [FS_251]
- Limit [LIM_250] (rows=100 width=496)
+ File Output Operator [FS_286]
+ Limit [LIM_285] (rows=100 width=496)
Number of rows:100
- Select Operator [SEL_249] (rows=4156223234 width=496)
+ Select Operator [SEL_284] (rows=4156223234 width=496)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
<-Reducer 6 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_248]
- Group By Operator [GBY_247] (rows=4156223234 width=496)
+ SHUFFLE [RS_283]
+ Group By Operator [GBY_282] (rows=4156223234 width=496)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_47]
@@ -142,124 +142,124 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["sum(_col10)","sum(_col16)","sum(_col3)"],keys:_col22, _col23, _col19, _col20
Top N Key Operator [TNK_96] (rows=4156223234 width=483)
keys:_col22, _col23, _col19, _col20,top n:100
- Merge Join Operator [MERGEJOIN_214] (rows=4156223234 width=483)
- Conds:RS_42._col6=RS_246._col0(Inner),Output:["_col3","_col10","_col16","_col19","_col20","_col22","_col23"]
+ Merge Join Operator [MERGEJOIN_249] (rows=4156223234 width=483)
+ Conds:RS_42._col6=RS_281._col0(Inner),Output:["_col3","_col10","_col16","_col19","_col20","_col22","_col23"]
<-Map 17 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_246]
+ SHUFFLE [RS_281]
PartitionCols:_col0
- Select Operator [SEL_245] (rows=462000 width=288)
+ Select Operator [SEL_280] (rows=462000 width=288)
Output:["_col0","_col1","_col2"]
TableScan [TS_31] (rows=462000 width=288)
default@item,item,Tbl:COMPLETE,Col:COMPLETE,Output:["i_item_sk","i_item_id","i_item_desc"]
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_42]
PartitionCols:_col6
- Merge Join Operator [MERGEJOIN_213] (rows=4156223234 width=203)
- Conds:RS_39._col8=RS_244._col0(Inner),Output:["_col3","_col6","_col10","_col16","_col19","_col20"]
+ Merge Join Operator [MERGEJOIN_248] (rows=4156223234 width=203)
+ Conds:RS_39._col8=RS_279._col0(Inner),Output:["_col3","_col6","_col10","_col16","_col19","_col20"]
<-Map 16 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_244]
+ SHUFFLE [RS_279]
PartitionCols:_col0
- Select Operator [SEL_243] (rows=1704 width=192)
+ Select Operator [SEL_278] (rows=1704 width=192)
Output:["_col0","_col1","_col2"]
TableScan [TS_29] (rows=1704 width=192)
default@store,store,Tbl:COMPLETE,Col:COMPLETE,Output:["s_store_sk","s_store_id","s_store_name"]
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_39]
PartitionCols:_col8
- Merge Join Operator [MERGEJOIN_212] (rows=4156223234 width=19)
+ Merge Join Operator [MERGEJOIN_247] (rows=4156223234 width=19)
Conds:RS_36._col1, _col2=RS_37._col9, _col8(Inner),Output:["_col3","_col6","_col8","_col10","_col16"]
<-Reducer 11 [SIMPLE_EDGE]
SHUFFLE [RS_37]
PartitionCols:_col9, _col8
- Merge Join Operator [MERGEJOIN_211] (rows=21091879 width=18)
+ Merge Join Operator [MERGEJOIN_246] (rows=21091879 width=18)
Conds:RS_25._col2, _col1, _col4=RS_26._col2, _col1, _col3(Inner),Output:["_col1","_col3","_col5","_col8","_col9","_col11"]
<-Reducer 10 [SIMPLE_EDGE]
SHUFFLE [RS_25]
PartitionCols:_col2, _col1, _col4
- Merge Join Operator [MERGEJOIN_209] (rows=13737330 width=8)
- Conds:RS_239._col0=RS_223._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5"]
+ Merge Join Operator [MERGEJOIN_244] (rows=13737330 width=8)
+ Conds:RS_274._col0=RS_258._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5"]
<-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_223]
+ PARTITION_ONLY_SHUFFLE [RS_258]
PartitionCols:_col0
- Select Operator [SEL_219] (rows=50 width=4)
+ Select Operator [SEL_254] (rows=50 width=4)
Output:["_col0"]
- Filter Operator [FIL_216] (rows=50 width=12)
+ Filter Operator [FIL_251] (rows=50 width=12)
predicate:((d_year = 1999) and (d_moy = 4))
TableScan [TS_3] (rows=73049 width=8)
default@date_dim,d3,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_year","d_moy"]
<-Map 14 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_239]
+ SHUFFLE [RS_274]
PartitionCols:_col0
- Select Operator [SEL_238] (rows=501694138 width=23)
+ Select Operator [SEL_273] (rows=501694138 width=23)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_237] (rows=501694138 width=23)
+ Filter Operator [FIL_272] (rows=501694138 width=23)
predicate:(ss_sold_date_sk is not null and ss_customer_sk is not null and ss_store_sk is not null and ss_sold_date_sk BETWEEN DynamicValue(RS_23_d1_d_date_sk_min) AND DynamicValue(RS_23_d1_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_23_d1_d_date_sk_bloom_filter)))
TableScan [TS_6] (rows=575995635 width=23)
default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_item_sk","ss_customer_sk","ss_store_sk","ss_ticket_number","ss_quantity"]
<-Reducer 12 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_236]
- Group By Operator [GBY_235] (rows=1 width=12)
+ BROADCAST [RS_271]
+ Group By Operator [GBY_270] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_229]
- Group By Operator [GBY_227] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_264]
+ Group By Operator [GBY_262] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_224] (rows=50 width=4)
+ Select Operator [SEL_259] (rows=50 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_219]
+ Please refer to the previous Select Operator [SEL_254]
<-Reducer 13 [SIMPLE_EDGE]
SHUFFLE [RS_26]
PartitionCols:_col2, _col1, _col3
- Merge Join Operator [MERGEJOIN_210] (rows=5384572 width=13)
- Conds:RS_242._col0=RS_225._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_245] (rows=5384572 width=13)
+ Conds:RS_277._col0=RS_260._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
<-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_225]
+ PARTITION_ONLY_SHUFFLE [RS_260]
PartitionCols:_col0
- Select Operator [SEL_220] (rows=201 width=4)
+ Select Operator [SEL_255] (rows=201 width=4)
Output:["_col0"]
- Filter Operator [FIL_217] (rows=201 width=12)
+ Filter Operator [FIL_252] (rows=201 width=12)
predicate:((d_year = 1999) and d_moy BETWEEN 4 AND 7)
Please refer to the previous TableScan [TS_3]
<-Map 15 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_242]
+ SHUFFLE [RS_277]
PartitionCols:_col0
- Select Operator [SEL_241] (rows=53632139 width=19)
+ Select Operator [SEL_276] (rows=53632139 width=19)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_240] (rows=53632139 width=19)
+ Filter Operator [FIL_275] (rows=53632139 width=19)
predicate:(sr_customer_sk is not null and sr_returned_date_sk is not null)
TableScan [TS_12] (rows=57591150 width=19)
default@store_returns,store_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["sr_returned_date_sk","sr_item_sk","sr_customer_sk","sr_ticket_number","sr_return_quantity"]
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_36]
PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_208] (rows=7638375 width=10)
- Conds:RS_234._col0=RS_221._col0(Inner),Output:["_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_243] (rows=7638375 width=10)
+ Conds:RS_269._col0=RS_256._col0(Inner),Output:["_col1","_col2","_col3"]
<-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_221]
+ PARTITION_ONLY_SHUFFLE [RS_256]
PartitionCols:_col0
- Select Operator [SEL_218] (rows=1957 width=4)
+ Select Operator [SEL_253] (rows=1957 width=4)
Output:["_col0"]
- Filter Operator [FIL_215] (rows=1957 width=8)
+ Filter Operator [FIL_250] (rows=1957 width=8)
predicate:(d_year) IN (1999, 2000, 2001)
Please refer to the previous TableScan [TS_3]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_234]
+ SHUFFLE [RS_269]
PartitionCols:_col0
- Select Operator [SEL_233] (rows=285117831 width=15)
+ Select Operator [SEL_268] (rows=285117831 width=15)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_232] (rows=285117831 width=15)
+ Filter Operator [FIL_267] (rows=285117831 width=15)
predicate:(cs_sold_date_sk is not null and cs_bill_customer_sk is not null and cs_sold_date_sk BETWEEN DynamicValue(RS_34_d3_d_date_sk_min) AND DynamicValue(RS_34_d3_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_34_d3_d_date_sk_bloom_filter)))
TableScan [TS_0] (rows=287989836 width=15)
default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["cs_sold_date_sk","cs_bill_customer_sk","cs_item_sk","cs_quantity"]
<-Reducer 9 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_231]
- Group By Operator [GBY_230] (rows=1 width=12)
+ BROADCAST [RS_266]
+ Group By Operator [GBY_265] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_228]
- Group By Operator [GBY_226] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_263]
+ Group By Operator [GBY_261] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_222] (rows=1957 width=4)
+ Select Operator [SEL_257] (rows=1957 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_218]
+ Please refer to the previous Select Operator [SEL_253]
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query39.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query39.q.out
index 07b1872..f9af51f 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query39.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query39.q.out
@@ -77,23 +77,23 @@ Stage-0
limit:-1
Stage-1
Reducer 6 vectorized
- File Output Operator [FS_166]
- Select Operator [SEL_165] (rows=859 width=56)
+ File Output Operator [FS_176]
+ Select Operator [SEL_175] (rows=859 width=56)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_44]
Select Operator [SEL_43] (rows=859 width=48)
Output:["_col0","_col1","_col2","_col3","_col6","_col7"]
- Merge Join Operator [MERGEJOIN_144] (rows=859 width=40)
- Conds:RS_160._col0, _col1=RS_164._col0, _col1(Inner),Output:["_col0","_col1","_col2","_col3","_col6","_col7"]
+ Merge Join Operator [MERGEJOIN_154] (rows=859 width=40)
+ Conds:RS_170._col0, _col1=RS_174._col0, _col1(Inner),Output:["_col0","_col1","_col2","_col3","_col6","_col7"]
<-Reducer 4 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_160]
+ SHUFFLE [RS_170]
PartitionCols:_col0, _col1
- Select Operator [SEL_159] (rows=859 width=24)
+ Select Operator [SEL_169] (rows=859 width=24)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_158] (rows=859 width=40)
+ Filter Operator [FIL_168] (rows=859 width=40)
predicate:CASE WHEN (((UDFToDouble(_col2) / _col3) = 0)) THEN (false) ELSE (((power(((_col4 - ((_col5 * _col5) / _col3)) / CASE WHEN ((_col3 = 1L)) THEN (null) ELSE ((_col3 - 1)) END), 0.5) / (UDFToDouble(_col2) / _col3)) > 1.0D)) END
- Group By Operator [GBY_157] (rows=1719 width=40)
+ Group By Operator [GBY_167] (rows=1719 width=40)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0, KEY._col1
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_16]
@@ -102,44 +102,44 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col3)","count(_col3)","sum(_col5)","sum(_col4)"],keys:_col1, _col2
Select Operator [SEL_13] (rows=1032514 width=8)
Output:["_col1","_col2","_col3","_col4","_col5"]
- Merge Join Operator [MERGEJOIN_141] (rows=1032514 width=8)
- Conds:RS_10._col1=RS_155._col0(Inner),Output:["_col2","_col3","_col5"]
+ Merge Join Operator [MERGEJOIN_151] (rows=1032514 width=8)
+ Conds:RS_10._col1=RS_165._col0(Inner),Output:["_col2","_col3","_col5"]
<-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_155]
+ SHUFFLE [RS_165]
PartitionCols:_col0
- Select Operator [SEL_154] (rows=27 width=4)
+ Select Operator [SEL_164] (rows=27 width=4)
Output:["_col0"]
TableScan [TS_5] (rows=27 width=4)
default@warehouse,warehouse,Tbl:COMPLETE,Col:COMPLETE,Output:["w_warehouse_sk"]
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_10]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_140] (rows=1032514 width=8)
- Conds:RS_146._col0=RS_152._col0(Inner),Output:["_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_150] (rows=1032514 width=8)
+ Conds:RS_156._col0=RS_162._col0(Inner),Output:["_col1","_col2","_col3"]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_146]
+ SHUFFLE [RS_156]
PartitionCols:_col0
- Select Operator [SEL_145] (rows=37584000 width=15)
+ Select Operator [SEL_155] (rows=37584000 width=15)
Output:["_col0","_col1","_col2","_col3"]
TableScan [TS_0] (rows=37584000 width=15)
default@inventory,inventory,Tbl:COMPLETE,Col:COMPLETE,Output:["inv_date_sk","inv_item_sk","inv_warehouse_sk","inv_quantity_on_hand"]
<-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_152]
+ SHUFFLE [RS_162]
PartitionCols:_col0
- Select Operator [SEL_150] (rows=50 width=4)
+ Select Operator [SEL_160] (rows=50 width=4)
Output:["_col0"]
- Filter Operator [FIL_148] (rows=50 width=12)
+ Filter Operator [FIL_158] (rows=50 width=12)
predicate:((d_year = 1999) and (d_moy = 5))
TableScan [TS_2] (rows=73049 width=12)
default@date_dim,date_dim,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_year","d_moy"]
<-Reducer 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_164]
+ SHUFFLE [RS_174]
PartitionCols:_col0, _col1
- Select Operator [SEL_163] (rows=859 width=24)
+ Select Operator [SEL_173] (rows=859 width=24)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_162] (rows=859 width=40)
+ Filter Operator [FIL_172] (rows=859 width=40)
predicate:CASE WHEN (((UDFToDouble(_col2) / _col3) = 0)) THEN (false) ELSE (((power(((_col4 - ((_col5 * _col5) / _col3)) / CASE WHEN ((_col3 = 1L)) THEN (null) ELSE ((_col3 - 1)) END), 0.5) / (UDFToDouble(_col2) / _col3)) > 1.0D)) END
- Group By Operator [GBY_161] (rows=1719 width=40)
+ Group By Operator [GBY_171] (rows=1719 width=40)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0, KEY._col1
<-Reducer 8 [SIMPLE_EDGE]
SHUFFLE [RS_36]
@@ -148,27 +148,27 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col3)","count(_col3)","sum(_col5)","sum(_col4)"],keys:_col1, _col2
Select Operator [SEL_33] (rows=1032514 width=8)
Output:["_col1","_col2","_col3","_col4","_col5"]
- Merge Join Operator [MERGEJOIN_143] (rows=1032514 width=8)
- Conds:RS_30._col1=RS_156._col0(Inner),Output:["_col2","_col3","_col5"]
+ Merge Join Operator [MERGEJOIN_153] (rows=1032514 width=8)
+ Conds:RS_30._col1=RS_166._col0(Inner),Output:["_col2","_col3","_col5"]
<-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_156]
+ SHUFFLE [RS_166]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_154]
+ Please refer to the previous Select Operator [SEL_164]
<-Reducer 7 [SIMPLE_EDGE]
SHUFFLE [RS_30]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_142] (rows=1032514 width=8)
- Conds:RS_147._col0=RS_153._col0(Inner),Output:["_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_152] (rows=1032514 width=8)
+ Conds:RS_157._col0=RS_163._col0(Inner),Output:["_col1","_col2","_col3"]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_147]
+ SHUFFLE [RS_157]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_145]
+ Please refer to the previous Select Operator [SEL_155]
<-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_153]
+ SHUFFLE [RS_163]
PartitionCols:_col0
- Select Operator [SEL_151] (rows=50 width=4)
+ Select Operator [SEL_161] (rows=50 width=4)
Output:["_col0"]
- Filter Operator [FIL_149] (rows=50 width=12)
+ Filter Operator [FIL_159] (rows=50 width=12)
predicate:((d_year = 1999) and (d_moy = 4))
Please refer to the previous TableScan [TS_2]
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query40.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query40.q.out
index a44ad59..5691fde 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query40.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query40.q.out
@@ -81,14 +81,14 @@ Stage-0
limit:100
Stage-1
Reducer 7 vectorized
- File Output Operator [FS_127]
- Limit [LIM_126] (rows=100 width=410)
+ File Output Operator [FS_132]
+ Limit [LIM_131] (rows=100 width=410)
Number of rows:100
- Select Operator [SEL_125] (rows=769995 width=410)
+ Select Operator [SEL_130] (rows=769995 width=410)
Output:["_col0","_col1","_col2","_col3"]
<-Reducer 6 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_124]
- Group By Operator [GBY_123] (rows=769995 width=410)
+ SHUFFLE [RS_129]
+ Group By Operator [GBY_128] (rows=769995 width=410)
Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0, KEY._col1
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_28]
@@ -99,72 +99,72 @@ Stage-0
Output:["_col0","_col1","_col2","_col3"]
Top N Key Operator [TNK_56] (rows=5757278 width=278)
keys:_col14, _col12,top n:100
- Merge Join Operator [MERGEJOIN_104] (rows=5757278 width=278)
- Conds:RS_22._col1=RS_122._col0(Inner),Output:["_col4","_col7","_col9","_col10","_col12","_col14"]
+ Merge Join Operator [MERGEJOIN_109] (rows=5757278 width=278)
+ Conds:RS_22._col1=RS_127._col0(Inner),Output:["_col4","_col7","_col9","_col10","_col12","_col14"]
<-Map 12 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_122]
+ SHUFFLE [RS_127]
PartitionCols:_col0
- Select Operator [SEL_121] (rows=27 width=90)
+ Select Operator [SEL_126] (rows=27 width=90)
Output:["_col0","_col1"]
TableScan [TS_11] (rows=27 width=90)
default@warehouse,warehouse,Tbl:COMPLETE,Col:COMPLETE,Output:["w_warehouse_sk","w_state"]
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_22]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_103] (rows=5757278 width=195)
- Conds:RS_19._col2=RS_107._col0(Inner),Output:["_col1","_col4","_col7","_col9","_col10","_col12"]
+ Merge Join Operator [MERGEJOIN_108] (rows=5757278 width=195)
+ Conds:RS_19._col2=RS_112._col0(Inner),Output:["_col1","_col4","_col7","_col9","_col10","_col12"]
<-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_107]
+ SHUFFLE [RS_112]
PartitionCols:_col0
- Select Operator [SEL_106] (rows=51333 width=104)
+ Select Operator [SEL_111] (rows=51333 width=104)
Output:["_col0","_col1"]
- Filter Operator [FIL_105] (rows=51333 width=215)
+ Filter Operator [FIL_110] (rows=51333 width=215)
predicate:i_current_price BETWEEN 0.99 AND 1.49
TableScan [TS_8] (rows=462000 width=215)
default@item,item,Tbl:COMPLETE,Col:COMPLETE,Output:["i_item_sk","i_item_id","i_current_price"]
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_19]
PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_102] (rows=51815831 width=124)
- Conds:RS_16._col0=RS_120._col0(Inner),Output:["_col1","_col2","_col4","_col7","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_107] (rows=51815831 width=124)
+ Conds:RS_16._col0=RS_125._col0(Inner),Output:["_col1","_col2","_col4","_col7","_col9","_col10"]
<-Map 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_120]
+ SHUFFLE [RS_125]
PartitionCols:_col0
- Select Operator [SEL_119] (rows=8116 width=12)
+ Select Operator [SEL_124] (rows=8116 width=12)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_118] (rows=8116 width=98)
+ Filter Operator [FIL_123] (rows=8116 width=98)
predicate:CAST( d_date AS TIMESTAMP) BETWEEN TIMESTAMP'1998-03-09 00:00:00' AND TIMESTAMP'1998-05-08 00:00:00'
TableScan [TS_5] (rows=73049 width=98)
default@date_dim,date_dim,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_date"]
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_16]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_101] (rows=466374405 width=167)
- Conds:RS_115._col2, _col3=RS_117._col0, _col1(Left Outer),Output:["_col0","_col1","_col2","_col4","_col7"]
+ Merge Join Operator [MERGEJOIN_106] (rows=466374405 width=167)
+ Conds:RS_120._col2, _col3=RS_122._col0, _col1(Left Outer),Output:["_col0","_col1","_col2","_col4","_col7"]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_115]
+ SHUFFLE [RS_120]
PartitionCols:_col2, _col3
- Select Operator [SEL_114] (rows=285115816 width=127)
+ Select Operator [SEL_119] (rows=285115816 width=127)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_113] (rows=285115816 width=127)
+ Filter Operator [FIL_118] (rows=285115816 width=127)
predicate:(cs_warehouse_sk is not null and cs_sold_date_sk is not null and cs_item_sk BETWEEN DynamicValue(RS_20_item_i_item_sk_min) AND DynamicValue(RS_20_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_20_item_i_item_sk_bloom_filter)))
TableScan [TS_0] (rows=287989836 width=127)
default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["cs_sold_date_sk","cs_warehouse_sk","cs_item_sk","cs_order_number","cs_sales_price"]
<-Reducer 11 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_112]
- Group By Operator [GBY_111] (rows=1 width=12)
+ BROADCAST [RS_117]
+ Group By Operator [GBY_116] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_110]
- Group By Operator [GBY_109] (rows=1 width=12)
+ SHUFFLE [RS_115]
+ Group By Operator [GBY_114] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_108] (rows=51333 width=4)
+ Select Operator [SEL_113] (rows=51333 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_106]
+ Please refer to the previous Select Operator [SEL_111]
<-Map 8 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_117]
+ SHUFFLE [RS_122]
PartitionCols:_col0, _col1
- Select Operator [SEL_116] (rows=28798881 width=117)
+ Select Operator [SEL_121] (rows=28798881 width=117)
Output:["_col0","_col1","_col2"]
TableScan [TS_3] (rows=28798881 width=117)
default@catalog_returns,catalog_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["cr_item_sk","cr_order_number","cr_refunded_cash"]
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query49.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query49.q.out
index 459060f..034590a 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query49.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query49.q.out
@@ -299,46 +299,46 @@ Stage-0
limit:100
Stage-1
Reducer 11 vectorized
- File Output Operator [FS_315]
- Limit [LIM_314] (rows=100 width=215)
+ File Output Operator [FS_345]
+ Limit [LIM_344] (rows=100 width=215)
Number of rows:100
- Select Operator [SEL_313] (rows=3418 width=215)
+ Select Operator [SEL_343] (rows=3418 width=215)
Output:["_col0","_col1","_col2","_col3","_col4"]
<-Reducer 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_312]
- Select Operator [SEL_311] (rows=3418 width=215)
+ SHUFFLE [RS_342]
+ Select Operator [SEL_341] (rows=3418 width=215)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_310] (rows=3418 width=215)
+ Group By Operator [GBY_340] (rows=3418 width=215)
Output:["_col0","_col1","_col2","_col3","_col4"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4
<-Union 9 [SIMPLE_EDGE]
<-Reducer 24 [CONTAINS] vectorized
- Reduce Output Operator [RS_356]
+ Reduce Output Operator [RS_386]
PartitionCols:_col0, _col1, _col2, _col3, _col4
- Group By Operator [GBY_355] (rows=3418 width=215)
+ Group By Operator [GBY_385] (rows=3418 width=215)
Output:["_col0","_col1","_col2","_col3","_col4"],keys:_col0, _col3, _col4, _col1, _col2
- Top N Key Operator [TNK_354] (rows=3418 width=214)
+ Top N Key Operator [TNK_384] (rows=3418 width=214)
keys:_col0, _col3, _col4, _col1, _col2,top n:100
- Select Operator [SEL_353] (rows=1142 width=213)
+ Select Operator [SEL_383] (rows=1142 width=213)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_352] (rows=1142 width=248)
+ Filter Operator [FIL_382] (rows=1142 width=248)
predicate:((_col0 <= 10) or (rank_window_1 <= 10))
- PTF Operator [PTF_351] (rows=1714 width=248)
+ PTF Operator [PTF_381] (rows=1714 width=248)
Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col4 AS decimal(15,4)) / CAST( _col5 AS decimal(15,4))) ASC NULLS LAST","partition by:":"0"}]
- Select Operator [SEL_350] (rows=1714 width=248)
+ Select Operator [SEL_380] (rows=1714 width=248)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
<-Reducer 23 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_349]
+ SHUFFLE [RS_379]
PartitionCols:0
- Select Operator [SEL_348] (rows=1714 width=244)
+ Select Operator [SEL_378] (rows=1714 width=244)
Output:["rank_window_0","_col0","_col1","_col2","_col3","_col4"]
- PTF Operator [PTF_347] (rows=1714 width=244)
+ PTF Operator [PTF_377] (rows=1714 width=244)
Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col1 AS decimal(15,4)) / CAST( _col2 AS decimal(15,4))) ASC NULLS LAST","partition by:":"0"}]
- Select Operator [SEL_346] (rows=1714 width=244)
+ Select Operator [SEL_376] (rows=1714 width=244)
Output:["_col0","_col1","_col2","_col3","_col4"]
<-Reducer 22 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_345]
+ SHUFFLE [RS_375]
PartitionCols:0
- Group By Operator [GBY_344] (rows=1714 width=244)
+ Group By Operator [GBY_374] (rows=1714 width=244)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
<-Reducer 21 [SIMPLE_EDGE]
SHUFFLE [RS_89]
@@ -347,89 +347,89 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0
Select Operator [SEL_86] (rows=1673571 width=73)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Merge Join Operator [MERGEJOIN_242] (rows=1673571 width=73)
- Conds:RS_83._col1, _col2=RS_343._col0, _col1(Inner),Output:["_col1","_col3","_col4","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_272] (rows=1673571 width=73)
+ Conds:RS_83._col1, _col2=RS_373._col0, _col1(Inner),Output:["_col1","_col3","_col4","_col9","_col10"]
<-Map 30 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_343]
+ SHUFFLE [RS_373]
PartitionCols:_col0, _col1
- Select Operator [SEL_342] (rows=19197050 width=119)
+ Select Operator [SEL_372] (rows=19197050 width=119)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_341] (rows=19197050 width=119)
+ Filter Operator [FIL_371] (rows=19197050 width=119)
predicate:(sr_return_amt > 10000)
TableScan [TS_77] (rows=57591150 width=119)
default@store_returns,sr,Tbl:COMPLETE,Col:COMPLETE,Output:["sr_item_sk","sr_ticket_number","sr_return_quantity","sr_return_amt"]
<-Reducer 20 [SIMPLE_EDGE]
SHUFFLE [RS_83]
PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_241] (rows=1673571 width=8)
- Conds:RS_340._col0=RS_277._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_271] (rows=1673571 width=8)
+ Conds:RS_370._col0=RS_307._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
<-Map 12 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_277]
+ PARTITION_ONLY_SHUFFLE [RS_307]
PartitionCols:_col0
- Select Operator [SEL_272] (rows=50 width=4)
+ Select Operator [SEL_302] (rows=50 width=4)
Output:["_col0"]
- Filter Operator [FIL_271] (rows=50 width=12)
+ Filter Operator [FIL_301] (rows=50 width=12)
predicate:((d_year = 2000) and (d_moy = 12))
TableScan [TS_3] (rows=73049 width=12)
default@date_dim,date_dim,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_year","d_moy"]
<-Map 29 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_340]
+ SHUFFLE [RS_370]
PartitionCols:_col0
- Select Operator [SEL_339] (rows=61119617 width=229)
+ Select Operator [SEL_369] (rows=61119617 width=229)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_338] (rows=61119617 width=229)
+ Filter Operator [FIL_368] (rows=61119617 width=229)
predicate:((ss_quantity > 0) and (ss_net_profit > 1) and (ss_net_paid > 0) and ss_sold_date_sk is not null and ss_sold_date_sk BETWEEN DynamicValue(RS_81_date_dim_d_date_sk_min) AND DynamicValue(RS_81_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_81_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_71] (rows=575995635 width=229)
default@store_sales,sts,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_item_sk","ss_ticket_number","ss_quantity","ss_net_paid","ss_net_profit"]
<-Reducer 25 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_337]
- Group By Operator [GBY_336] (rows=1 width=12)
+ BROADCAST [RS_367]
+ Group By Operator [GBY_366] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_284]
- Group By Operator [GBY_281] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_314]
+ Group By Operator [GBY_311] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_278] (rows=50 width=4)
+ Select Operator [SEL_308] (rows=50 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_272]
+ Please refer to the previous Select Operator [SEL_302]
<-Reducer 8 [CONTAINS] vectorized
- Reduce Output Operator [RS_309]
+ Reduce Output Operator [RS_339]
PartitionCols:_col0, _col1, _col2, _col3, _col4
- Group By Operator [GBY_308] (rows=3418 width=215)
+ Group By Operator [GBY_338] (rows=3418 width=215)
Output:["_col0","_col1","_col2","_col3","_col4"],keys:_col0, _col3, _col4, _col1, _col2
- Top N Key Operator [TNK_307] (rows=3418 width=214)
+ Top N Key Operator [TNK_337] (rows=3418 width=214)
keys:_col0, _col3, _col4, _col1, _col2,top n:100
- Select Operator [SEL_306] (rows=2276 width=215)
+ Select Operator [SEL_336] (rows=2276 width=215)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_305] (rows=2276 width=215)
+ Group By Operator [GBY_335] (rows=2276 width=215)
Output:["_col0","_col1","_col2","_col3","_col4"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4
<-Union 7 [SIMPLE_EDGE]
<-Reducer 18 [CONTAINS] vectorized
- Reduce Output Operator [RS_335]
+ Reduce Output Operator [RS_365]
PartitionCols:_col0, _col1, _col2, _col3, _col4
- Group By Operator [GBY_334] (rows=2276 width=215)
+ Group By Operator [GBY_364] (rows=2276 width=215)
Output:["_col0","_col1","_col2","_col3","_col4"],keys:_col0, _col3, _col4, _col1, _col2
- Select Operator [SEL_333] (rows=1134 width=215)
+ Select Operator [SEL_363] (rows=1134 width=215)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_332] (rows=1134 width=248)
+ Filter Operator [FIL_362] (rows=1134 width=248)
predicate:((_col0 <= 10) or (rank_window_1 <= 10))
- PTF Operator [PTF_331] (rows=1701 width=248)
+ PTF Operator [PTF_361] (rows=1701 width=248)
Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col4 AS decimal(15,4)) / CAST( _col5 AS decimal(15,4))) ASC NULLS LAST","partition by:":"0"}]
- Select Operator [SEL_330] (rows=1701 width=248)
+ Select Operator [SEL_360] (rows=1701 width=248)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
<-Reducer 17 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_329]
+ SHUFFLE [RS_359]
PartitionCols:0
- Select Operator [SEL_328] (rows=1701 width=244)
+ Select Operator [SEL_358] (rows=1701 width=244)
Output:["rank_window_0","_col0","_col1","_col2","_col3","_col4"]
- PTF Operator [PTF_327] (rows=1701 width=244)
+ PTF Operator [PTF_357] (rows=1701 width=244)
Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col1 AS decimal(15,4)) / CAST( _col2 AS decimal(15,4))) ASC NULLS LAST","partition by:":"0"}]
- Select Operator [SEL_326] (rows=1701 width=244)
+ Select Operator [SEL_356] (rows=1701 width=244)
Output:["_col0","_col1","_col2","_col3","_col4"]
<-Reducer 16 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_325]
+ SHUFFLE [RS_355]
PartitionCols:0
- Group By Operator [GBY_324] (rows=1701 width=244)
+ Group By Operator [GBY_354] (rows=1701 width=244)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
<-Reducer 15 [SIMPLE_EDGE]
SHUFFLE [RS_50]
@@ -438,72 +438,72 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0
Select Operator [SEL_47] (rows=865646 width=188)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Merge Join Operator [MERGEJOIN_240] (rows=865646 width=188)
- Conds:RS_44._col1, _col2=RS_323._col0, _col1(Inner),Output:["_col1","_col3","_col4","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_270] (rows=865646 width=188)
+ Conds:RS_44._col1, _col2=RS_353._col0, _col1(Inner),Output:["_col1","_col3","_col4","_col9","_col10"]
<-Map 28 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_323]
+ SHUFFLE [RS_353]
PartitionCols:_col0, _col1
- Select Operator [SEL_322] (rows=9599627 width=121)
+ Select Operator [SEL_352] (rows=9599627 width=121)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_321] (rows=9599627 width=121)
+ Filter Operator [FIL_351] (rows=9599627 width=121)
predicate:(cr_return_amount > 10000)
TableScan [TS_38] (rows=28798881 width=121)
default@catalog_returns,cr,Tbl:COMPLETE,Col:COMPLETE,Output:["cr_item_sk","cr_order_number","cr_return_quantity","cr_return_amount"]
<-Reducer 14 [SIMPLE_EDGE]
SHUFFLE [RS_44]
PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_239] (rows=865646 width=102)
- Conds:RS_320._col0=RS_275._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_269] (rows=865646 width=102)
+ Conds:RS_350._col0=RS_305._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
<-Map 12 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_275]
+ PARTITION_ONLY_SHUFFLE [RS_305]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_272]
+ Please refer to the previous Select Operator [SEL_302]
<-Map 27 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_320]
+ SHUFFLE [RS_350]
PartitionCols:_col0
- Select Operator [SEL_319] (rows=31838858 width=239)
+ Select Operator [SEL_349] (rows=31838858 width=239)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_318] (rows=31838858 width=239)
+ Filter Operator [FIL_348] (rows=31838858 width=239)
predicate:((cs_quantity > 0) and (cs_net_profit > 1) and (cs_net_paid > 0) and cs_sold_date_sk is not null and cs_sold_date_sk BETWEEN DynamicValue(RS_42_date_dim_d_date_sk_min) AND DynamicValue(RS_42_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_42_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_32] (rows=287989836 width=239)
default@catalog_sales,cs,Tbl:COMPLETE,Col:COMPLETE,Output:["cs_sold_date_sk","cs_item_sk","cs_order_number","cs_quantity","cs_net_paid","cs_net_profit"]
<-Reducer 19 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_317]
- Group By Operator [GBY_316] (rows=1 width=12)
+ BROADCAST [RS_347]
+ Group By Operator [GBY_346] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_283]
- Group By Operator [GBY_280] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_313]
+ Group By Operator [GBY_310] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_276] (rows=50 width=4)
+ Select Operator [SEL_306] (rows=50 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_272]
+ Please refer to the previous Select Operator [SEL_302]
<-Reducer 6 [CONTAINS] vectorized
- Reduce Output Operator [RS_304]
+ Reduce Output Operator [RS_334]
PartitionCols:_col0, _col1, _col2, _col3, _col4
- Group By Operator [GBY_303] (rows=2276 width=215)
+ Group By Operator [GBY_333] (rows=2276 width=215)
Output:["_col0","_col1","_col2","_col3","_col4"],keys:_col0, _col3, _col4, _col1, _col2
- Select Operator [SEL_302] (rows=1142 width=211)
+ Select Operator [SEL_332] (rows=1142 width=211)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_301] (rows=1142 width=248)
+ Filter Operator [FIL_331] (rows=1142 width=248)
predicate:((_col0 <= 10) or (rank_window_1 <= 10))
- PTF Operator [PTF_300] (rows=1714 width=248)
+ PTF Operator [PTF_330] (rows=1714 width=248)
Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col4 AS decimal(15,4)) / CAST( _col5 AS decimal(15,4))) ASC NULLS LAST","partition by:":"0"}]
- Select Operator [SEL_299] (rows=1714 width=248)
+ Select Operator [SEL_329] (rows=1714 width=248)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
<-Reducer 5 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_298]
+ SHUFFLE [RS_328]
PartitionCols:0
- Select Operator [SEL_297] (rows=1714 width=244)
+ Select Operator [SEL_327] (rows=1714 width=244)
Output:["rank_window_0","_col0","_col1","_col2","_col3","_col4"]
- PTF Operator [PTF_296] (rows=1714 width=244)
+ PTF Operator [PTF_326] (rows=1714 width=244)
Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col1 AS decimal(15,4)) / CAST( _col2 AS decimal(15,4))) ASC NULLS LAST","partition by:":"0"}]
- Select Operator [SEL_295] (rows=1714 width=244)
+ Select Operator [SEL_325] (rows=1714 width=244)
Output:["_col0","_col1","_col2","_col3","_col4"]
<-Reducer 4 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_294]
+ SHUFFLE [RS_324]
PartitionCols:0
- Group By Operator [GBY_293] (rows=1714 width=244)
+ Group By Operator [GBY_323] (rows=1714 width=244)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_18]
@@ -512,44 +512,44 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0
Select Operator [SEL_15] (rows=438010 width=177)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Merge Join Operator [MERGEJOIN_238] (rows=438010 width=177)
- Conds:RS_12._col1, _col2=RS_292._col0, _col1(Inner),Output:["_col1","_col3","_col4","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_268] (rows=438010 width=177)
+ Conds:RS_12._col1, _col2=RS_322._col0, _col1(Inner),Output:["_col1","_col3","_col4","_col9","_col10"]
<-Map 26 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_292]
+ SHUFFLE [RS_322]
PartitionCols:_col0, _col1
- Select Operator [SEL_291] (rows=4799489 width=118)
+ Select Operator [SEL_321] (rows=4799489 width=118)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_290] (rows=4799489 width=118)
+ Filter Operator [FIL_320] (rows=4799489 width=118)
predicate:(wr_return_amt > 10000)
TableScan [TS_6] (rows=14398467 width=118)
default@web_returns,wr,Tbl:COMPLETE,Col:COMPLETE,Output:["wr_item_sk","wr_order_number","wr_return_quantity","wr_return_amt"]
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_12]
PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_237] (rows=438010 width=122)
- Conds:RS_289._col0=RS_273._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_267] (rows=438010 width=122)
+ Conds:RS_319._col0=RS_303._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
<-Map 12 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_273]
+ PARTITION_ONLY_SHUFFLE [RS_303]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_272]
+ Please refer to the previous Select Operator [SEL_302]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_289]
+ SHUFFLE [RS_319]
PartitionCols:_col0
- Select Operator [SEL_288] (rows=15996318 width=239)
+ Select Operator [SEL_318] (rows=15996318 width=239)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_287] (rows=15996318 width=239)
+ Filter Operator [FIL_317] (rows=15996318 width=239)
predicate:((ws_quantity > 0) and (ws_net_profit > 1) and (ws_net_paid > 0) and ws_sold_date_sk is not null and ws_sold_date_sk BETWEEN DynamicValue(RS_10_date_dim_d_date_sk_min) AND DynamicValue(RS_10_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_10_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_0] (rows=144002668 width=239)
default@web_sales,ws,Tbl:COMPLETE,Col:COMPLETE,Output:["ws_sold_date_sk","ws_item_sk","ws_order_number","ws_quantity","ws_net_paid","ws_net_profit"]
<-Reducer 13 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_286]
- Group By Operator [GBY_285] (rows=1 width=12)
+ BROADCAST [RS_316]
+ Group By Operator [GBY_315] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_282]
- Group By Operator [GBY_279] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_312]
+ Group By Operator [GBY_309] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_274] (rows=50 width=4)
+ Select Operator [SEL_304] (rows=50 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_272]
+ Please refer to the previous Select Operator [SEL_302]
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query5.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query5.q.out
index a5870f7..bfc1fac 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query5.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query5.q.out
@@ -281,6 +281,7 @@ Map 1 <- Reducer 11 (BROADCAST_EDGE), Union 2 (CONTAINS)
Map 21 <- Reducer 15 (BROADCAST_EDGE), Union 22 (CONTAINS)
Map 23 <- Union 22 (CONTAINS)
Map 25 <- Reducer 19 (BROADCAST_EDGE), Union 26 (CONTAINS)
+Map 27 <- Reducer 30 (BROADCAST_EDGE)
Map 9 <- Union 2 (CONTAINS)
Reducer 11 <- Map 10 (CUSTOM_SIMPLE_EDGE)
Reducer 12 <- Map 10 (SIMPLE_EDGE), Union 22 (SIMPLE_EDGE)
@@ -288,11 +289,12 @@ Reducer 13 <- Map 24 (SIMPLE_EDGE), Reducer 12 (SIMPLE_EDGE)
Reducer 14 <- Reducer 13 (SIMPLE_EDGE), Union 6 (CONTAINS)
Reducer 15 <- Map 10 (CUSTOM_SIMPLE_EDGE)
Reducer 16 <- Map 10 (SIMPLE_EDGE), Union 26 (SIMPLE_EDGE)
-Reducer 17 <- Map 30 (SIMPLE_EDGE), Reducer 16 (SIMPLE_EDGE)
+Reducer 17 <- Map 31 (SIMPLE_EDGE), Reducer 16 (SIMPLE_EDGE)
Reducer 18 <- Reducer 17 (SIMPLE_EDGE), Union 6 (CONTAINS)
Reducer 19 <- Map 10 (CUSTOM_SIMPLE_EDGE)
Reducer 28 <- Map 27 (SIMPLE_EDGE), Map 29 (SIMPLE_EDGE), Union 26 (CONTAINS)
Reducer 3 <- Map 10 (SIMPLE_EDGE), Union 2 (SIMPLE_EDGE)
+Reducer 30 <- Map 29 (CUSTOM_SIMPLE_EDGE)
Reducer 4 <- Map 20 (SIMPLE_EDGE), Reducer 3 (SIMPLE_EDGE)
Reducer 5 <- Reducer 4 (SIMPLE_EDGE), Union 6 (CONTAINS)
Reducer 7 <- Union 6 (SIMPLE_EDGE)
@@ -303,229 +305,240 @@ Stage-0
limit:100
Stage-1
Reducer 8 vectorized
- File Output Operator [FS_305]
- Limit [LIM_304] (rows=100 width=619)
+ File Output Operator [FS_315]
+ Limit [LIM_314] (rows=100 width=619)
Number of rows:100
- Select Operator [SEL_303] (rows=59581 width=619)
+ Select Operator [SEL_313] (rows=59581 width=619)
Output:["_col0","_col1","_col2","_col3","_col4"]
<-Reducer 7 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_302]
- Select Operator [SEL_301] (rows=59581 width=619)
+ SHUFFLE [RS_312]
+ Select Operator [SEL_311] (rows=59581 width=619)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_300] (rows=59581 width=627)
+ Group By Operator [GBY_310] (rows=59581 width=627)
Output:["_col0","_col1","_col3","_col4","_col5"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Union 6 [SIMPLE_EDGE]
<-Reducer 14 [CONTAINS] vectorized
- Reduce Output Operator [RS_315]
+ Reduce Output Operator [RS_325]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_314] (rows=59581 width=627)
+ Group By Operator [GBY_324] (rows=59581 width=627)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0, _col1, 0L
- Top N Key Operator [TNK_313] (rows=39721 width=618)
+ Top N Key Operator [TNK_323] (rows=39721 width=618)
keys:_col0, _col1,top n:100
- Select Operator [SEL_312] (rows=38846 width=619)
+ Select Operator [SEL_322] (rows=38846 width=619)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_311] (rows=38846 width=548)
+ Group By Operator [GBY_321] (rows=38846 width=548)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
<-Reducer 13 [SIMPLE_EDGE]
SHUFFLE [RS_45]
PartitionCols:_col0
Group By Operator [GBY_44] (rows=2835758 width=548)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col2)","sum(_col4)","sum(_col3)","sum(_col5)"],keys:_col8
- Merge Join Operator [MERGEJOIN_224] (rows=34813117 width=535)
- Conds:RS_40._col0=RS_310._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col8"]
+ Merge Join Operator [MERGEJOIN_234] (rows=34813117 width=535)
+ Conds:RS_40._col0=RS_320._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col8"]
<-Map 24 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_310]
+ SHUFFLE [RS_320]
PartitionCols:_col0
- Select Operator [SEL_309] (rows=46000 width=104)
+ Select Operator [SEL_319] (rows=46000 width=104)
Output:["_col0","_col1"]
TableScan [TS_35] (rows=46000 width=104)
default@catalog_page,catalog_page,Tbl:COMPLETE,Col:COMPLETE,Output:["cp_catalog_page_sk","cp_catalog_page_id"]
<-Reducer 12 [SIMPLE_EDGE]
SHUFFLE [RS_40]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_223] (rows=34813117 width=438)
- Conds:Union 22._col1=RS_278._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5"]
+ Merge Join Operator [MERGEJOIN_233] (rows=34813117 width=438)
+ Conds:Union 22._col1=RS_288._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5"]
<-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_278]
+ SHUFFLE [RS_288]
PartitionCols:_col0
- Select Operator [SEL_275] (rows=8116 width=4)
+ Select Operator [SEL_285] (rows=8116 width=4)
Output:["_col0"]
- Filter Operator [FIL_274] (rows=8116 width=98)
+ Filter Operator [FIL_284] (rows=8116 width=98)
predicate:CAST( d_date AS TIMESTAMP) BETWEEN TIMESTAMP'1998-08-04 00:00:00' AND TIMESTAMP'1998-08-18 00:00:00'
TableScan [TS_8] (rows=73049 width=98)
default@date_dim,date_dim,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_date"]
<-Union 22 [SIMPLE_EDGE]
<-Map 21 [CONTAINS] vectorized
- Reduce Output Operator [RS_327]
+ Reduce Output Operator [RS_337]
PartitionCols:_col1
- Select Operator [SEL_326] (rows=285117694 width=455)
+ Select Operator [SEL_336] (rows=285117694 width=455)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_325] (rows=285117694 width=231)
+ Filter Operator [FIL_335] (rows=285117694 width=231)
predicate:(cs_sold_date_sk is not null and cs_catalog_page_sk is not null and cs_sold_date_sk BETWEEN DynamicValue(RS_38_date_dim_d_date_sk_min) AND DynamicValue(RS_38_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_38_date_dim_d_date_sk_bloom_filter)))
- TableScan [TS_255] (rows=287989836 width=231)
+ TableScan [TS_265] (rows=287989836 width=231)
Output:["cs_sold_date_sk","cs_catalog_page_sk","cs_ext_sales_price","cs_net_profit"]
<-Reducer 15 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_324]
- Group By Operator [GBY_323] (rows=1 width=12)
+ BROADCAST [RS_334]
+ Group By Operator [GBY_333] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_286]
- Group By Operator [GBY_283] (rows=1 width=12)
+ SHUFFLE [RS_296]
+ Group By Operator [GBY_293] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_279] (rows=8116 width=4)
+ Select Operator [SEL_289] (rows=8116 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_275]
+ Please refer to the previous Select Operator [SEL_285]
<-Map 23 [CONTAINS] vectorized
- Reduce Output Operator [RS_330]
+ Reduce Output Operator [RS_340]
PartitionCols:_col1
- Select Operator [SEL_329] (rows=28221805 width=451)
+ Select Operator [SEL_339] (rows=28221805 width=451)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_328] (rows=28221805 width=227)
+ Filter Operator [FIL_338] (rows=28221805 width=227)
predicate:(cr_catalog_page_sk is not null and cr_returned_date_sk is not null)
- TableScan [TS_260] (rows=28798881 width=227)
+ TableScan [TS_270] (rows=28798881 width=227)
Output:["cr_returned_date_sk","cr_catalog_page_sk","cr_return_amount","cr_net_loss"]
<-Reducer 18 [CONTAINS] vectorized
- Reduce Output Operator [RS_322]
+ Reduce Output Operator [RS_332]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_321] (rows=59581 width=627)
+ Group By Operator [GBY_331] (rows=59581 width=627)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0, _col1, 0L
- Top N Key Operator [TNK_320] (rows=39721 width=618)
+ Top N Key Operator [TNK_330] (rows=39721 width=618)
keys:_col0, _col1,top n:100
- Select Operator [SEL_319] (rows=53 width=615)
+ Select Operator [SEL_329] (rows=53 width=615)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_318] (rows=53 width=548)
+ Group By Operator [GBY_328] (rows=53 width=548)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
<-Reducer 17 [SIMPLE_EDGE]
SHUFFLE [RS_77]
PartitionCols:_col0
Group By Operator [GBY_76] (rows=3498 width=548)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col2)","sum(_col4)","sum(_col3)","sum(_col5)"],keys:_col8
- Merge Join Operator [MERGEJOIN_226] (rows=30966059 width=543)
- Conds:RS_72._col0=RS_317._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col8"]
- <-Map 30 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_317]
+ Merge Join Operator [MERGEJOIN_236] (rows=30966059 width=543)
+ Conds:RS_72._col0=RS_327._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col8"]
+ <-Map 31 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_327]
PartitionCols:_col0
- Select Operator [SEL_316] (rows=84 width=104)
+ Select Operator [SEL_326] (rows=84 width=104)
Output:["_col0","_col1"]
TableScan [TS_67] (rows=84 width=104)
default@web_site,web_site,Tbl:COMPLETE,Col:COMPLETE,Output:["web_site_sk","web_site_id"]
<-Reducer 16 [SIMPLE_EDGE]
SHUFFLE [RS_72]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_225] (rows=30966059 width=447)
- Conds:Union 26._col1=RS_280._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5"]
+ Merge Join Operator [MERGEJOIN_235] (rows=30966059 width=447)
+ Conds:Union 26._col1=RS_290._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5"]
<-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_280]
+ SHUFFLE [RS_290]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_275]
+ Please refer to the previous Select Operator [SEL_285]
<-Union 26 [SIMPLE_EDGE]
<-Map 25 [CONTAINS] vectorized
- Reduce Output Operator [RS_335]
+ Reduce Output Operator [RS_345]
PartitionCols:_col1
- Select Operator [SEL_334] (rows=143930874 width=455)
+ Select Operator [SEL_344] (rows=143930874 width=455)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_333] (rows=143930874 width=231)
+ Filter Operator [FIL_343] (rows=143930874 width=231)
predicate:(ws_web_site_sk is not null and ws_sold_date_sk is not null and ws_sold_date_sk BETWEEN DynamicValue(RS_70_date_dim_d_date_sk_min) AND DynamicValue(RS_70_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_70_date_dim_d_date_sk_bloom_filter)))
- TableScan [TS_265] (rows=144002668 width=231)
+ TableScan [TS_275] (rows=144002668 width=231)
Output:["ws_sold_date_sk","ws_web_site_sk","ws_ext_sales_price","ws_net_profit"]
<-Reducer 19 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_332]
- Group By Operator [GBY_331] (rows=1 width=12)
+ BROADCAST [RS_342]
+ Group By Operator [GBY_341] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_287]
- Group By Operator [GBY_284] (rows=1 width=12)
+ SHUFFLE [RS_297]
+ Group By Operator [GBY_294] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_281] (rows=8116 width=4)
+ Select Operator [SEL_291] (rows=8116 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_275]
+ Please refer to the previous Select Operator [SEL_285]
<-Reducer 28 [CONTAINS]
- Reduce Output Operator [RS_273]
+ Reduce Output Operator [RS_283]
PartitionCols:_col1
- Select Operator [SEL_271] (rows=134782734 width=454)
+ Select Operator [SEL_281] (rows=134782734 width=454)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Merge Join Operator [MERGEJOIN_270] (rows=134782734 width=230)
- Conds:RS_338._col0, _col2=RS_341._col1, _col2(Inner),Output:["_col1","_col3","_col6","_col7"]
- <-Map 27 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_338]
- PartitionCols:_col0, _col2
- Select Operator [SEL_337] (rows=143966669 width=11)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_336] (rows=143966669 width=11)
- predicate:ws_web_site_sk is not null
- TableScan [TS_52] (rows=144002668 width=11)
- default@web_sales,web_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ws_item_sk","ws_web_site_sk","ws_order_number"]
+ Merge Join Operator [MERGEJOIN_280] (rows=134782734 width=230)
+ Conds:RS_356._col0, _col2=RS_348._col1, _col2(Inner),Output:["_col1","_col3","_col6","_col7"]
<-Map 29 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_341]
+ PARTITION_ONLY_SHUFFLE [RS_348]
PartitionCols:_col1, _col2
- Select Operator [SEL_340] (rows=13749816 width=225)
+ Select Operator [SEL_347] (rows=13749816 width=225)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_339] (rows=13749816 width=225)
+ Filter Operator [FIL_346] (rows=13749816 width=225)
predicate:wr_returned_date_sk is not null
TableScan [TS_55] (rows=14398467 width=225)
default@web_returns,web_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["wr_returned_date_sk","wr_item_sk","wr_order_number","wr_return_amt","wr_net_loss"]
+ <-Map 27 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_356]
+ PartitionCols:_col0, _col2
+ Select Operator [SEL_355] (rows=143966669 width=11)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_354] (rows=143966669 width=11)
+ predicate:(ws_web_site_sk is not null and ws_item_sk BETWEEN DynamicValue(RS[224]_col0) AND DynamicValue(RS[224]_col1) and ws_order_number BETWEEN DynamicValue(RS[224]_col2) AND DynamicValue(RS[224]_col3) and in_bloom_filter(hash(ws_item_sk,ws_order_number), DynamicValue(RS[224]_col4)))
+ TableScan [TS_52] (rows=144002668 width=11)
+ default@web_sales,web_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ws_item_sk","ws_web_site_sk","ws_order_number"]
+ <-Reducer 30 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_353]
+ Group By Operator [GBY_352] (rows=1 width=160)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","min(VALUE._col2)","max(VALUE._col3)","bloom_filter(VALUE._col4, expectedEntries=8007986)"]
+ <-Map 29 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_351]
+ Group By Operator [GBY_350] (rows=1 width=160)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["min(_col0)","max(_col0)","min(_col1)","max(_col1)","bloom_filter(_col3, expectedEntries=8007986)"]
+ Select Operator [SEL_349] (rows=13749816 width=12)
+ Output:["_col0","_col1","_col3"]
+ Please refer to the previous Select Operator [SEL_347]
<-Reducer 5 [CONTAINS] vectorized
- Reduce Output Operator [RS_299]
+ Reduce Output Operator [RS_309]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_298] (rows=59581 width=627)
+ Group By Operator [GBY_308] (rows=59581 width=627)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0, _col1, 0L
- Top N Key Operator [TNK_297] (rows=39721 width=618)
+ Top N Key Operator [TNK_307] (rows=39721 width=618)
keys:_col0, _col1,top n:100
- Select Operator [SEL_296] (rows=822 width=617)
+ Select Operator [SEL_306] (rows=822 width=617)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_295] (rows=822 width=548)
+ Group By Operator [GBY_305] (rows=822 width=548)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_21]
PartitionCols:_col0
Group By Operator [GBY_20] (rows=78090 width=548)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col2)","sum(_col4)","sum(_col3)","sum(_col5)"],keys:_col8
- Merge Join Operator [MERGEJOIN_222] (rows=64325014 width=376)
- Conds:RS_16._col0=RS_294._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col8"]
+ Merge Join Operator [MERGEJOIN_232] (rows=64325014 width=376)
+ Conds:RS_16._col0=RS_304._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col8"]
<-Map 20 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_294]
+ SHUFFLE [RS_304]
PartitionCols:_col0
- Select Operator [SEL_293] (rows=1704 width=104)
+ Select Operator [SEL_303] (rows=1704 width=104)
Output:["_col0","_col1"]
TableScan [TS_11] (rows=1704 width=104)
default@store,store,Tbl:COMPLETE,Col:COMPLETE,Output:["s_store_sk","s_store_id"]
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_16]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_221] (rows=64325014 width=277)
- Conds:Union 2._col1=RS_276._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5"]
+ Merge Join Operator [MERGEJOIN_231] (rows=64325014 width=277)
+ Conds:Union 2._col1=RS_286._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5"]
<-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_276]
+ SHUFFLE [RS_286]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_275]
+ Please refer to the previous Select Operator [SEL_285]
<-Union 2 [SIMPLE_EDGE]
<-Map 1 [CONTAINS] vectorized
- Reduce Output Operator [RS_292]
+ Reduce Output Operator [RS_302]
PartitionCols:_col1
- Select Operator [SEL_291] (rows=525329897 width=445)
+ Select Operator [SEL_301] (rows=525329897 width=445)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_290] (rows=525329897 width=221)
+ Filter Operator [FIL_300] (rows=525329897 width=221)
predicate:(ss_sold_date_sk is not null and ss_store_sk is not null and ss_sold_date_sk BETWEEN DynamicValue(RS_14_date_dim_d_date_sk_min) AND DynamicValue(RS_14_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_14_date_dim_d_date_sk_bloom_filter)))
- TableScan [TS_227] (rows=575995635 width=221)
+ TableScan [TS_237] (rows=575995635 width=221)
Output:["ss_sold_date_sk","ss_store_sk","ss_ext_sales_price","ss_net_profit"]
<-Reducer 11 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_289]
- Group By Operator [GBY_288] (rows=1 width=12)
+ BROADCAST [RS_299]
+ Group By Operator [GBY_298] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_285]
- Group By Operator [GBY_282] (rows=1 width=12)
+ SHUFFLE [RS_295]
+ Group By Operator [GBY_292] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_277] (rows=8116 width=4)
+ Select Operator [SEL_287] (rows=8116 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_275]
+ Please refer to the previous Select Operator [SEL_285]
<-Map 9 [CONTAINS] vectorized
- Reduce Output Operator [RS_308]
+ Reduce Output Operator [RS_318]
PartitionCols:_col1
- Select Operator [SEL_307] (rows=53634860 width=447)
+ Select Operator [SEL_317] (rows=53634860 width=447)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_306] (rows=53634860 width=223)
+ Filter Operator [FIL_316] (rows=53634860 width=223)
predicate:(sr_store_sk is not null and sr_returned_date_sk is not null)
- TableScan [TS_238] (rows=57591150 width=223)
+ TableScan [TS_248] (rows=57591150 width=223)
Output:["sr_returned_date_sk","sr_store_sk","sr_return_amt","sr_net_loss"]
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query50.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query50.q.out
index 7d0ffc7..f2864f8 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query50.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query50.q.out
@@ -140,14 +140,14 @@ Stage-0
limit:100
Stage-1
Reducer 6 vectorized
- File Output Operator [FS_119]
- Limit [LIM_118] (rows=100 width=858)
+ File Output Operator [FS_129]
+ Limit [LIM_128] (rows=100 width=858)
Number of rows:100
- Select Operator [SEL_117] (rows=11945216 width=857)
+ Select Operator [SEL_127] (rows=11945216 width=857)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"]
<-Reducer 5 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_116]
- Group By Operator [GBY_115] (rows=11945216 width=857)
+ SHUFFLE [RS_126]
+ Group By Operator [GBY_125] (rows=11945216 width=857)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)","sum(VALUE._col4)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5, KEY._col6, KEY._col7, KEY._col8, KEY._col9
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_23]
@@ -158,61 +158,61 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"]
Top N Key Operator [TNK_46] (rows=11945216 width=821)
keys:_col11, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20,top n:100
- Merge Join Operator [MERGEJOIN_101] (rows=11945216 width=821)
- Conds:RS_17._col8=RS_114._col0(Inner),Output:["_col0","_col5","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20"]
+ Merge Join Operator [MERGEJOIN_111] (rows=11945216 width=821)
+ Conds:RS_17._col8=RS_124._col0(Inner),Output:["_col0","_col5","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20"]
<-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_114]
+ SHUFFLE [RS_124]
PartitionCols:_col0
- Select Operator [SEL_113] (rows=1704 width=821)
+ Select Operator [SEL_123] (rows=1704 width=821)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10"]
TableScan [TS_9] (rows=1704 width=821)
default@store,store,Tbl:COMPLETE,Col:COMPLETE,Output:["s_store_sk","s_store_name","s_company_id","s_street_number","s_street_name","s_street_type","s_suite_number","s_city","s_county","s_state","s_zip"]
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_17]
PartitionCols:_col8
- Merge Join Operator [MERGEJOIN_100] (rows=11945216 width=3)
- Conds:RS_14._col1, _col2, _col3=RS_112._col1, _col2, _col4(Inner),Output:["_col0","_col5","_col8"]
+ Merge Join Operator [MERGEJOIN_110] (rows=11945216 width=3)
+ Conds:RS_14._col1, _col2, _col3=RS_122._col1, _col2, _col4(Inner),Output:["_col0","_col5","_col8"]
<-Reducer 2 [SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_14]
PartitionCols:_col1, _col2, _col3
- Merge Join Operator [MERGEJOIN_99] (rows=1339446 width=8)
- Conds:RS_104._col0=RS_107._col0(Inner),Output:["_col0","_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_109] (rows=1339446 width=8)
+ Conds:RS_114._col0=RS_117._col0(Inner),Output:["_col0","_col1","_col2","_col3"]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_104]
+ SHUFFLE [RS_114]
PartitionCols:_col0
- Select Operator [SEL_103] (rows=53632139 width=15)
+ Select Operator [SEL_113] (rows=53632139 width=15)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_102] (rows=53632139 width=15)
+ Filter Operator [FIL_112] (rows=53632139 width=15)
predicate:(sr_customer_sk is not null and sr_returned_date_sk is not null)
TableScan [TS_0] (rows=57591150 width=15)
default@store_returns,store_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["sr_returned_date_sk","sr_item_sk","sr_customer_sk","sr_ticket_number"]
<-Map 8 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_107]
+ SHUFFLE [RS_117]
PartitionCols:_col0
- Select Operator [SEL_106] (rows=50 width=4)
+ Select Operator [SEL_116] (rows=50 width=4)
Output:["_col0"]
- Filter Operator [FIL_105] (rows=50 width=12)
+ Filter Operator [FIL_115] (rows=50 width=12)
predicate:((d_year = 2000) and (d_moy = 9))
TableScan [TS_3] (rows=73049 width=12)
default@date_dim,d2,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_year","d_moy"]
<-Map 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_112]
+ SHUFFLE [RS_122]
PartitionCols:_col1, _col2, _col4
- Select Operator [SEL_111] (rows=501694138 width=19)
+ Select Operator [SEL_121] (rows=501694138 width=19)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_110] (rows=501694138 width=19)
- predicate:(ss_sold_date_sk is not null and ss_customer_sk is not null and ss_store_sk is not null and ss_ticket_number BETWEEN DynamicValue(RS_14_store_returns_sr_ticket_number_min) AND DynamicValue(RS_14_store_returns_sr_ticket_number_max) and in_bloom_filter(ss_ticket_number, DynamicValue(RS_14_store_returns_sr_ticket_number_bloom_filter)))
+ Filter Operator [FIL_120] (rows=501694138 width=19)
+ predicate:(ss_sold_date_sk is not null and ss_customer_sk is not null and ss_store_sk is not null and ss_item_sk BETWEEN DynamicValue(RS[108]_col0) AND DynamicValue(RS[108]_col1) and ss_customer_sk BETWEEN DynamicValue(RS[108]_col2) AND DynamicValue(RS[108]_col3) and ss_ticket_number BETWEEN DynamicValue(RS[108]_col4) AND DynamicValue(RS[108]_col5) and in_bloom_filter(hash(ss_item_sk,ss_customer_sk,ss_ticket_number), DynamicValue(RS[108]_col6)))
TableScan [TS_6] (rows=575995635 width=19)
default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_item_sk","ss_customer_sk","ss_store_sk","ss_ticket_number"]
<-Reducer 7 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_109]
- Group By Operator [GBY_108] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ BROADCAST [RS_119]
+ Group By Operator [GBY_118] (rows=1 width=168)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","min(VALUE._col2)","max(VALUE._col3)","min(VALUE._col4)","max(VALUE._col5)","bloom_filter(VALUE._col6, expectedEntries=1000000)"]
<-Reducer 2 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_86]
- Group By Operator [GBY_85] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_84] (rows=1339446 width=8)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_99]
+ PARTITION_ONLY_SHUFFLE [RS_106]
+ Group By Operator [GBY_105] (rows=1 width=168)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["min(_col0)","max(_col0)","min(_col1)","max(_col1)","min(_col2)","max(_col2)","bloom_filter(_col4, expectedEntries=1000000)"]
+ Select Operator [SEL_104] (rows=1339446 width=12)
+ Output:["_col0","_col1","_col2","_col4"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_109]
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query54.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query54.q.out
index 84dd405..d672c61 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query54.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query54.q.out
@@ -1,7 +1,7 @@
-Warning: Shuffle Join MERGEJOIN[279][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
-Warning: Shuffle Join MERGEJOIN[285][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
-Warning: Shuffle Join MERGEJOIN[284][tables = [$hdt$_4, $hdt$_5]] in Stage 'Reducer 15' is a cross product
-Warning: Shuffle Join MERGEJOIN[287][tables = [$hdt$_4, $hdt$_5, $hdt$_6]] in Stage 'Reducer 16' is a cross product
+Warning: Shuffle Join MERGEJOIN[289][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[295][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[294][tables = [$hdt$_4, $hdt$_5]] in Stage 'Reducer 15' is a cross product
+Warning: Shuffle Join MERGEJOIN[297][tables = [$hdt$_4, $hdt$_5, $hdt$_6]] in Stage 'Reducer 16' is a cross product
PREHOOK: query: explain
with my_customers as (
select distinct c_customer_sk
@@ -163,36 +163,36 @@ Stage-0
limit:100
Stage-1
Reducer 8 vectorized
- File Output Operator [FS_373]
- Limit [LIM_372] (rows=1 width=16)
+ File Output Operator [FS_383]
+ Limit [LIM_382] (rows=1 width=16)
Number of rows:100
- Select Operator [SEL_371] (rows=1 width=16)
+ Select Operator [SEL_381] (rows=1 width=16)
Output:["_col0","_col1","_col2"]
<-Reducer 7 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_370]
- Select Operator [SEL_369] (rows=1 width=16)
+ SHUFFLE [RS_380]
+ Select Operator [SEL_379] (rows=1 width=16)
Output:["_col0","_col1","_col2"]
- Top N Key Operator [TNK_368] (rows=1 width=12)
+ Top N Key Operator [TNK_378] (rows=1 width=12)
keys:_col0, _col1,top n:100
- Group By Operator [GBY_367] (rows=1 width=12)
+ Group By Operator [GBY_377] (rows=1 width=12)
Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0
<-Reducer 6 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_366]
+ SHUFFLE [RS_376]
PartitionCols:_col0
- Group By Operator [GBY_365] (rows=1 width=12)
+ Group By Operator [GBY_375] (rows=1 width=12)
Output:["_col0","_col1"],aggregations:["count()"],keys:_col0
- Select Operator [SEL_364] (rows=1 width=116)
+ Select Operator [SEL_374] (rows=1 width=116)
Output:["_col0"]
- Top N Key Operator [TNK_363] (rows=1 width=116)
+ Top N Key Operator [TNK_373] (rows=1 width=116)
keys:UDFToInteger((_col1 / 50)),top n:100
- Group By Operator [GBY_362] (rows=1 width=116)
+ Group By Operator [GBY_372] (rows=1 width=116)
Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_119]
PartitionCols:_col0
Group By Operator [GBY_118] (rows=228 width=116)
Output:["_col0","_col1"],aggregations:["sum(_col2)"],keys:_col5
- Merge Join Operator [MERGEJOIN_289] (rows=525327388 width=110)
+ Merge Join Operator [MERGEJOIN_299] (rows=525327388 width=110)
Conds:RS_114._col0=RS_115._col0(Inner),Output:["_col2","_col5"]
<-Reducer 16 [SIMPLE_EDGE]
SHUFFLE [RS_115]
@@ -201,229 +201,229 @@ Stage-0
Output:["_col0"]
Filter Operator [FIL_103] (rows=5072841 width=12)
predicate:(_col1 <= _col3)
- Merge Join Operator [MERGEJOIN_287] (rows=15218525 width=12)
+ Merge Join Operator [MERGEJOIN_297] (rows=15218525 width=12)
Conds:(Inner),Output:["_col0","_col1","_col3"]
<-Reducer 15 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_100]
Filter Operator [FIL_99] (rows=608741 width=12)
predicate:(_col2 <= _col1)
- Merge Join Operator [MERGEJOIN_284] (rows=1826225 width=12)
+ Merge Join Operator [MERGEJOIN_294] (rows=1826225 width=12)
Conds:(Inner),Output:["_col0","_col1","_col2"]
<-Map 9 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_320]
- Select Operator [SEL_313] (rows=73049 width=8)
+ PARTITION_ONLY_SHUFFLE [RS_330]
+ Select Operator [SEL_323] (rows=73049 width=8)
Output:["_col0","_col1"]
- Filter Operator [FIL_307] (rows=73049 width=8)
+ Filter Operator [FIL_317] (rows=73049 width=8)
predicate:d_month_seq is not null
TableScan [TS_3] (rows=73049 width=12)
default@date_dim,date_dim,Tbl:COMPLETE,Col:COMPLETE,Output:["d_month_seq","d_year","d_moy","d_date_sk"]
<-Reducer 14 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_359]
- Group By Operator [GBY_358] (rows=25 width=4)
+ PARTITION_ONLY_SHUFFLE [RS_369]
+ Group By Operator [GBY_368] (rows=25 width=4)
Output:["_col0"],keys:KEY._col0
<-Map 9 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_323]
+ PARTITION_ONLY_SHUFFLE [RS_333]
PartitionCols:_col0
- Group By Operator [GBY_316] (rows=25 width=4)
+ Group By Operator [GBY_326] (rows=25 width=4)
Output:["_col0"],keys:_col0
- Select Operator [SEL_310] (rows=50 width=12)
+ Select Operator [SEL_320] (rows=50 width=12)
Output:["_col0"]
- Filter Operator [FIL_304] (rows=50 width=12)
+ Filter Operator [FIL_314] (rows=50 width=12)
predicate:((d_year = 1999) and (d_moy = 3) and d_month_seq is not null)
Please refer to the previous TableScan [TS_3]
<-Reducer 17 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_361]
- Group By Operator [GBY_360] (rows=25 width=4)
+ PARTITION_ONLY_SHUFFLE [RS_371]
+ Group By Operator [GBY_370] (rows=25 width=4)
Output:["_col0"],keys:KEY._col0
<-Map 9 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_324]
+ PARTITION_ONLY_SHUFFLE [RS_334]
PartitionCols:_col0
- Group By Operator [GBY_317] (rows=25 width=4)
+ Group By Operator [GBY_327] (rows=25 width=4)
Output:["_col0"],keys:_col0
- Select Operator [SEL_311] (rows=50 width=12)
+ Select Operator [SEL_321] (rows=50 width=12)
Output:["_col0"]
- Filter Operator [FIL_305] (rows=50 width=12)
+ Filter Operator [FIL_315] (rows=50 width=12)
predicate:((d_year = 1999) and (d_moy = 3) and d_month_seq is not null)
Please refer to the previous TableScan [TS_3]
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_114]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_288] (rows=525327388 width=114)
+ Merge Join Operator [MERGEJOIN_298] (rows=525327388 width=114)
Conds:RS_111._col1=RS_112._col0(Inner),Output:["_col0","_col2","_col5"]
<-Reducer 22 [SIMPLE_EDGE]
SHUFFLE [RS_112]
PartitionCols:_col0
Select Operator [SEL_76] (rows=55046 width=4)
Output:["_col0"]
- Merge Join Operator [MERGEJOIN_286] (rows=55046 width=4)
- Conds:RS_73._col0=RS_351._col1(Inner),Output:["_col5"]
+ Merge Join Operator [MERGEJOIN_296] (rows=55046 width=4)
+ Conds:RS_73._col0=RS_361._col1(Inner),Output:["_col5"]
<-Reducer 21 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_351]
+ SHUFFLE [RS_361]
PartitionCols:_col1
- Select Operator [SEL_350] (rows=55046 width=8)
+ Select Operator [SEL_360] (rows=55046 width=8)
Output:["_col0","_col1"]
- Group By Operator [GBY_349] (rows=55046 width=8)
+ Group By Operator [GBY_359] (rows=55046 width=8)
Output:["_col0","_col1"],keys:KEY._col0, KEY._col1
<-Reducer 20 [SIMPLE_EDGE]
SHUFFLE [RS_67]
PartitionCols:_col0, _col1
Group By Operator [GBY_66] (rows=55046 width=8)
Output:["_col0","_col1"],keys:_col6, _col5
- Merge Join Operator [MERGEJOIN_283] (rows=110092 width=8)
- Conds:RS_62._col1=RS_348._col0(Inner),Output:["_col5","_col6"]
+ Merge Join Operator [MERGEJOIN_293] (rows=110092 width=8)
+ Conds:RS_62._col1=RS_358._col0(Inner),Output:["_col5","_col6"]
<-Map 31 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_348]
+ SHUFFLE [RS_358]
PartitionCols:_col0
- Select Operator [SEL_347] (rows=80000000 width=8)
+ Select Operator [SEL_357] (rows=80000000 width=8)
Output:["_col0","_col1"]
- Filter Operator [FIL_346] (rows=80000000 width=8)
+ Filter Operator [FIL_356] (rows=80000000 width=8)
predicate:c_current_addr_sk is not null
TableScan [TS_53] (rows=80000000 width=8)
default@customer,customer,Tbl:COMPLETE,Col:COMPLETE,Output:["c_customer_sk","c_current_addr_sk"]
<-Reducer 19 [SIMPLE_EDGE]
SHUFFLE [RS_62]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_282] (rows=110092 width=0)
- Conds:RS_59._col2=RS_345._col0(Inner),Output:["_col1"]
+ Merge Join Operator [MERGEJOIN_292] (rows=110092 width=0)
+ Conds:RS_59._col2=RS_355._col0(Inner),Output:["_col1"]
<-Map 30 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_345]
+ SHUFFLE [RS_355]
PartitionCols:_col0
- Select Operator [SEL_344] (rows=453 width=4)
+ Select Operator [SEL_354] (rows=453 width=4)
Output:["_col0"]
- Filter Operator [FIL_343] (rows=453 width=186)
+ Filter Operator [FIL_353] (rows=453 width=186)
predicate:((i_class = 'consignment') and (i_category = 'Jewelry'))
TableScan [TS_50] (rows=462000 width=186)
default@item,item,Tbl:COMPLETE,Col:COMPLETE,Output:["i_item_sk","i_class","i_category"]
<-Reducer 18 [SIMPLE_EDGE]
SHUFFLE [RS_59]
PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_281] (rows=11665117 width=7)
- Conds:Union 28._col0=RS_318._col0(Inner),Output:["_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_291] (rows=11665117 width=7)
+ Conds:Union 28._col0=RS_328._col0(Inner),Output:["_col1","_col2"]
<-Map 9 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_318]
+ PARTITION_ONLY_SHUFFLE [RS_328]
PartitionCols:_col0
- Select Operator [SEL_312] (rows=50 width=4)
+ Select Operator [SEL_322] (rows=50 width=4)
Output:["_col0"]
- Filter Operator [FIL_306] (rows=50 width=12)
+ Filter Operator [FIL_316] (rows=50 width=12)
predicate:((d_year = 1999) and (d_moy = 3))
Please refer to the previous TableScan [TS_3]
<-Union 28 [SIMPLE_EDGE]
<-Map 27 [CONTAINS] vectorized
- Reduce Output Operator [RS_379]
+ Reduce Output Operator [RS_389]
PartitionCols:_col0
- Select Operator [SEL_378] (rows=285117831 width=11)
+ Select Operator [SEL_388] (rows=285117831 width=11)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_377] (rows=285117831 width=11)
+ Filter Operator [FIL_387] (rows=285117831 width=11)
predicate:(cs_sold_date_sk is not null and cs_bill_customer_sk is not null and cs_sold_date_sk BETWEEN DynamicValue(RS_57_date_dim_d_date_sk_min) AND DynamicValue(RS_57_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_57_date_dim_d_date_sk_bloom_filter)))
- TableScan [TS_290] (rows=287989836 width=11)
+ TableScan [TS_300] (rows=287989836 width=11)
Output:["cs_sold_date_sk","cs_bill_customer_sk","cs_item_sk"]
<-Reducer 23 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_375]
- Group By Operator [GBY_374] (rows=1 width=12)
+ BROADCAST [RS_385]
+ Group By Operator [GBY_384] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 9 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_326]
- Group By Operator [GBY_325] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_336]
+ Group By Operator [GBY_335] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_319] (rows=50 width=4)
+ Select Operator [SEL_329] (rows=50 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_312]
+ Please refer to the previous Select Operator [SEL_322]
<-Map 29 [CONTAINS] vectorized
- Reduce Output Operator [RS_382]
+ Reduce Output Operator [RS_392]
PartitionCols:_col0
- Select Operator [SEL_381] (rows=143930993 width=11)
+ Select Operator [SEL_391] (rows=143930993 width=11)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_380] (rows=143930993 width=11)
+ Filter Operator [FIL_390] (rows=143930993 width=11)
predicate:(ws_bill_customer_sk is not null and ws_sold_date_sk is not null and ws_sold_date_sk BETWEEN DynamicValue(RS_57_date_dim_d_date_sk_min) AND DynamicValue(RS_57_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_57_date_dim_d_date_sk_bloom_filter)))
- TableScan [TS_295] (rows=144002668 width=11)
+ TableScan [TS_305] (rows=144002668 width=11)
Output:["ws_sold_date_sk","ws_item_sk","ws_bill_customer_sk"]
<-Reducer 23 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_376]
- Please refer to the previous Group By Operator [GBY_374]
+ BROADCAST [RS_386]
+ Please refer to the previous Group By Operator [GBY_384]
<-Reducer 25 [SIMPLE_EDGE]
SHUFFLE [RS_73]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_280] (rows=39720279 width=4)
- Conds:RS_354._col1, _col2=RS_357._col0, _col1(Inner),Output:["_col0"]
+ Merge Join Operator [MERGEJOIN_290] (rows=39720279 width=4)
+ Conds:RS_364._col1, _col2=RS_367._col0, _col1(Inner),Output:["_col0"]
<-Map 24 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_354]
+ SHUFFLE [RS_364]
PartitionCols:_col1, _col2
- Select Operator [SEL_353] (rows=40000000 width=188)
+ Select Operator [SEL_363] (rows=40000000 width=188)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_352] (rows=40000000 width=188)
+ Filter Operator [FIL_362] (rows=40000000 width=188)
predicate:(ca_county is not null and ca_state is not null)
TableScan [TS_33] (rows=40000000 width=188)
default@customer_address,customer_address,Tbl:COMPLETE,Col:COMPLETE,Output:["ca_address_sk","ca_county","ca_state"]
<-Map 26 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_357]
+ SHUFFLE [RS_367]
PartitionCols:_col0, _col1
- Select Operator [SEL_356] (rows=1704 width=184)
+ Select Operator [SEL_366] (rows=1704 width=184)
Output:["_col0","_col1"]
- Filter Operator [FIL_355] (rows=1704 width=184)
+ Filter Operator [FIL_365] (rows=1704 width=184)
predicate:(s_county is not null and s_state is not null)
TableScan [TS_36] (rows=1704 width=184)
default@store,store,Tbl:COMPLETE,Col:COMPLETE,Output:["s_county","s_state"]
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_111]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_285] (rows=525327388 width=114)
+ Merge Join Operator [MERGEJOIN_295] (rows=525327388 width=114)
Conds:(Inner),Output:["_col0","_col1","_col2"]
<-Reducer 13 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_342]
- Select Operator [SEL_341] (rows=1 width=8)
- Filter Operator [FIL_340] (rows=1 width=8)
+ PARTITION_ONLY_SHUFFLE [RS_352]
+ Select Operator [SEL_351] (rows=1 width=8)
+ Filter Operator [FIL_350] (rows=1 width=8)
predicate:(sq_count_check(_col0) <= 1)
- Group By Operator [GBY_339] (rows=1 width=8)
+ Group By Operator [GBY_349] (rows=1 width=8)
Output:["_col0"],aggregations:["count(VALUE._col0)"]
<-Reducer 12 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_338]
- Group By Operator [GBY_337] (rows=1 width=8)
+ PARTITION_ONLY_SHUFFLE [RS_348]
+ Group By Operator [GBY_347] (rows=1 width=8)
Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_336] (rows=25 width=4)
- Group By Operator [GBY_335] (rows=25 width=4)
+ Select Operator [SEL_346] (rows=25 width=4)
+ Group By Operator [GBY_345] (rows=25 width=4)
Output:["_col0"],keys:KEY._col0
<-Map 9 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_322]
+ PARTITION_ONLY_SHUFFLE [RS_332]
PartitionCols:_col0
- Group By Operator [GBY_315] (rows=25 width=4)
+ Group By Operator [GBY_325] (rows=25 width=4)
Output:["_col0"],keys:_col0
- Select Operator [SEL_309] (rows=50 width=12)
+ Select Operator [SEL_319] (rows=50 width=12)
Output:["_col0"]
- Filter Operator [FIL_303] (rows=50 width=12)
+ Filter Operator [FIL_313] (rows=50 width=12)
predicate:((d_year = 1999) and (d_moy = 3))
Please refer to the previous TableScan [TS_3]
<-Reducer 2 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_108]
- Merge Join Operator [MERGEJOIN_279] (rows=525327388 width=114)
+ Merge Join Operator [MERGEJOIN_289] (rows=525327388 width=114)
Conds:(Inner),Output:["_col0","_col1","_col2"]
<-Map 1 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_302]
- Select Operator [SEL_301] (rows=525327388 width=114)
+ PARTITION_ONLY_SHUFFLE [RS_312]
+ Select Operator [SEL_311] (rows=525327388 width=114)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_300] (rows=525327388 width=114)
+ Filter Operator [FIL_310] (rows=525327388 width=114)
predicate:(ss_sold_date_sk is not null and ss_customer_sk is not null)
TableScan [TS_0] (rows=575995635 width=114)
default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_customer_sk","ss_ext_sales_price"]
<-Reducer 11 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_334]
- Select Operator [SEL_333] (rows=1 width=8)
- Filter Operator [FIL_332] (rows=1 width=8)
+ PARTITION_ONLY_SHUFFLE [RS_344]
+ Select Operator [SEL_343] (rows=1 width=8)
+ Filter Operator [FIL_342] (rows=1 width=8)
predicate:(sq_count_check(_col0) <= 1)
- Group By Operator [GBY_331] (rows=1 width=8)
+ Group By Operator [GBY_341] (rows=1 width=8)
Output:["_col0"],aggregations:["count(VALUE._col0)"]
<-Reducer 10 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_330]
- Group By Operator [GBY_329] (rows=1 width=8)
+ PARTITION_ONLY_SHUFFLE [RS_340]
+ Group By Operator [GBY_339] (rows=1 width=8)
Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_328] (rows=25 width=4)
- Group By Operator [GBY_327] (rows=25 width=4)
+ Select Operator [SEL_338] (rows=25 width=4)
+ Group By Operator [GBY_337] (rows=25 width=4)
Output:["_col0"],keys:KEY._col0
<-Map 9 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_321]
+ PARTITION_ONLY_SHUFFLE [RS_331]
PartitionCols:_col0
- Group By Operator [GBY_314] (rows=25 width=4)
+ Group By Operator [GBY_324] (rows=25 width=4)
Output:["_col0"],keys:_col0
- Select Operator [SEL_308] (rows=50 width=12)
+ Select Operator [SEL_318] (rows=50 width=12)
Output:["_col0"]
- Please refer to the previous Filter Operator [FIL_303]
+ Please refer to the previous Filter Operator [FIL_313]
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query64.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query64.q.out
index ac637c9..09d86ed 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query64.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query64.q.out
@@ -315,8 +315,8 @@ Stage-0
limit:-1
Stage-1
Reducer 18 vectorized
- File Output Operator [FS_1078]
- Select Operator [SEL_1077] (rows=104628491644 width=1702)
+ File Output Operator [FS_1128]
+ Select Operator [SEL_1127] (rows=104628491644 width=1702)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20"]
<-Reducer 17 [SIMPLE_EDGE]
SHUFFLE [RS_201]
@@ -324,451 +324,451 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18"]
Filter Operator [FIL_199] (rows=104628491644 width=1694)
predicate:(_col3 <= _col19)
- Merge Join Operator [MERGEJOIN_977] (rows=313885474933 width=1694)
- Conds:RS_1057._col1, _col0, _col2=RS_1076._col2, _col1, _col3(Inner),Output:["_col3","_col4","_col5","_col6","_col7","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22"]
+ Merge Join Operator [MERGEJOIN_1027] (rows=313885474933 width=1694)
+ Conds:RS_1107._col1, _col0, _col2=RS_1126._col2, _col1, _col3(Inner),Output:["_col3","_col4","_col5","_col6","_col7","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22"]
<-Reducer 16 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1057]
+ SHUFFLE [RS_1107]
PartitionCols:_col1, _col0, _col2
- Select Operator [SEL_1056] (rows=21304422 width=525)
+ Select Operator [SEL_1106] (rows=21304422 width=525)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- Filter Operator [FIL_1055] (rows=21304422 width=1255)
+ Filter Operator [FIL_1105] (rows=21304422 width=1255)
predicate:_col13 is not null
- Select Operator [SEL_1054] (rows=21304422 width=1255)
+ Select Operator [SEL_1104] (rows=21304422 width=1255)
Output:["_col0","_col1","_col2","_col13","_col14","_col15","_col16"]
- Group By Operator [GBY_1053] (rows=21304422 width=1255)
+ Group By Operator [GBY_1103] (rows=21304422 width=1255)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16"],aggregations:["count(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5, KEY._col6, KEY._col7, KEY._col8, KEY._col9, KEY._col10, KEY._col11, KEY._col12
<-Reducer 15 [SIMPLE_EDGE]
SHUFFLE [RS_93]
PartitionCols:_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12
Group By Operator [GBY_92] (rows=21304422 width=1255)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16"],aggregations:["count()","sum(_col8)","sum(_col9)","sum(_col10)"],keys:_col24, _col11, _col25, _col29, _col31, _col37, _col38, _col39, _col40, _col42, _col43, _col44, _col45
- Merge Join Operator [MERGEJOIN_961] (rows=21304422 width=1048)
- Conds:RS_88._col17=RS_1050._col0(Inner),Output:["_col8","_col9","_col10","_col11","_col24","_col25","_col29","_col31","_col37","_col38","_col39","_col40","_col42","_col43","_col44","_col45"]
+ Merge Join Operator [MERGEJOIN_1011] (rows=21304422 width=1048)
+ Conds:RS_88._col17=RS_1100._col0(Inner),Output:["_col8","_col9","_col10","_col11","_col24","_col25","_col29","_col31","_col37","_col38","_col39","_col40","_col42","_col43","_col44","_col45"]
<-Map 51 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1050]
+ SHUFFLE [RS_1100]
PartitionCols:_col0
- Select Operator [SEL_1048] (rows=40000000 width=365)
+ Select Operator [SEL_1098] (rows=40000000 width=365)
Output:["_col0","_col1","_col2","_col3","_col4"]
TableScan [TS_44] (rows=40000000 width=365)
default@customer_address,ad1,Tbl:COMPLETE,Col:COMPLETE,Output:["ca_address_sk","ca_street_number","ca_street_name","ca_city","ca_zip"]
<-Reducer 14 [SIMPLE_EDGE]
SHUFFLE [RS_88]
PartitionCols:_col17
- Merge Join Operator [MERGEJOIN_960] (rows=21304422 width=691)
- Conds:RS_85._col5=RS_1049._col0(Inner),Output:["_col8","_col9","_col10","_col11","_col17","_col24","_col25","_col29","_col31","_col37","_col38","_col39","_col40"]
+ Merge Join Operator [MERGEJOIN_1010] (rows=21304422 width=691)
+ Conds:RS_85._col5=RS_1099._col0(Inner),Output:["_col8","_col9","_col10","_col11","_col17","_col24","_col25","_col29","_col31","_col37","_col38","_col39","_col40"]
<-Map 51 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1049]
+ SHUFFLE [RS_1099]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1048]
+ Please refer to the previous Select Operator [SEL_1098]
<-Reducer 13 [SIMPLE_EDGE]
SHUFFLE [RS_85]
PartitionCols:_col5
Filter Operator [FIL_84] (rows=21304422 width=502)
predicate:(_col33 <> _col35)
- Merge Join Operator [MERGEJOIN_959] (rows=21304422 width=502)
- Conds:RS_81._col15=RS_1045._col0(Inner),Output:["_col5","_col8","_col9","_col10","_col11","_col17","_col24","_col25","_col29","_col31","_col33","_col35"]
+ Merge Join Operator [MERGEJOIN_1009] (rows=21304422 width=502)
+ Conds:RS_81._col15=RS_1095._col0(Inner),Output:["_col5","_col8","_col9","_col10","_col11","_col17","_col24","_col25","_col29","_col31","_col33","_col35"]
<-Map 50 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1045]
+ SHUFFLE [RS_1095]
PartitionCols:_col0
- Select Operator [SEL_1043] (rows=1861800 width=89)
+ Select Operator [SEL_1093] (rows=1861800 width=89)
Output:["_col0","_col1"]
TableScan [TS_40] (rows=1861800 width=89)
default@customer_demographics,cd1,Tbl:COMPLETE,Col:COMPLETE,Output:["cd_demo_sk","cd_marital_status"]
<-Reducer 12 [SIMPLE_EDGE]
SHUFFLE [RS_81]
PartitionCols:_col15
- Merge Join Operator [MERGEJOIN_958] (rows=21007353 width=418)
- Conds:RS_78._col3=RS_1044._col0(Inner),Output:["_col5","_col8","_col9","_col10","_col11","_col15","_col17","_col24","_col25","_col29","_col31","_col33"]
+ Merge Join Operator [MERGEJOIN_1008] (rows=21007353 width=418)
+ Conds:RS_78._col3=RS_1094._col0(Inner),Output:["_col5","_col8","_col9","_col10","_col11","_col15","_col17","_col24","_col25","_col29","_col31","_col33"]
<-Map 50 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1044]
+ SHUFFLE [RS_1094]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1043]
+ Please refer to the previous Select Operator [SEL_1093]
<-Reducer 11 [SIMPLE_EDGE]
SHUFFLE [RS_78]
PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_957] (rows=20714426 width=331)
- Conds:RS_75._col18=RS_984._col0(Inner),Output:["_col3","_col5","_col8","_col9","_col10","_col11","_col15","_col17","_col24","_col25","_col29","_col31"]
+ Merge Join Operator [MERGEJOIN_1007] (rows=20714426 width=331)
+ Conds:RS_75._col18=RS_1034._col0(Inner),Output:["_col3","_col5","_col8","_col9","_col10","_col11","_col15","_col17","_col24","_col25","_col29","_col31"]
<-Map 47 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_984]
+ PARTITION_ONLY_SHUFFLE [RS_1034]
PartitionCols:_col0
- Select Operator [SEL_978] (rows=73049 width=8)
+ Select Operator [SEL_1028] (rows=73049 width=8)
Output:["_col0","_col1"]
TableScan [TS_38] (rows=73049 width=8)
default@date_dim,d3,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_year"]
<-Reducer 10 [SIMPLE_EDGE]
SHUFFLE [RS_75]
PartitionCols:_col18
- Merge Join Operator [MERGEJOIN_956] (rows=20714426 width=331)
- Conds:RS_72._col19=RS_986._col0(Inner),Output:["_col3","_col5","_col8","_col9","_col10","_col11","_col15","_col17","_col18","_col24","_col25","_col29"]
+ Merge Join Operator [MERGEJOIN_1006] (rows=20714426 width=331)
+ Conds:RS_72._col19=RS_1036._col0(Inner),Output:["_col3","_col5","_col8","_col9","_col10","_col11","_col15","_col17","_col18","_col24","_col25","_col29"]
<-Map 47 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_986]
+ PARTITION_ONLY_SHUFFLE [RS_1036]
PartitionCols:_col0
- Select Operator [SEL_980] (rows=73049 width=8)
+ Select Operator [SEL_1030] (rows=73049 width=8)
Output:["_col0","_col1"]
Please refer to the previous TableScan [TS_38]
<-Reducer 9 [SIMPLE_EDGE]
SHUFFLE [RS_72]
PartitionCols:_col19
- Merge Join Operator [MERGEJOIN_955] (rows=20714426 width=330)
- Conds:RS_69._col16=RS_1040._col0(Inner),Output:["_col3","_col5","_col8","_col9","_col10","_col11","_col15","_col17","_col18","_col19","_col24","_col25"]
+ Merge Join Operator [MERGEJOIN_1005] (rows=20714426 width=330)
+ Conds:RS_69._col16=RS_1090._col0(Inner),Output:["_col3","_col5","_col8","_col9","_col10","_col11","_col15","_col17","_col18","_col19","_col24","_col25"]
<-Map 46 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1040]
+ SHUFFLE [RS_1090]
PartitionCols:_col0
- Select Operator [SEL_1038] (rows=7200 width=4)
+ Select Operator [SEL_1088] (rows=7200 width=4)
Output:["_col0"]
- Filter Operator [FIL_1037] (rows=7200 width=8)
+ Filter Operator [FIL_1087] (rows=7200 width=8)
predicate:hd_income_band_sk is not null
TableScan [TS_30] (rows=7200 width=8)
default@household_demographics,hd1,Tbl:COMPLETE,Col:COMPLETE,Output:["hd_demo_sk","hd_income_band_sk"]
<-Reducer 8 [SIMPLE_EDGE]
SHUFFLE [RS_69]
PartitionCols:_col16
- Merge Join Operator [MERGEJOIN_954] (rows=20714426 width=334)
- Conds:RS_66._col4=RS_1039._col0(Inner),Output:["_col3","_col5","_col8","_col9","_col10","_col11","_col15","_col16","_col17","_col18","_col19","_col24","_col25"]
+ Merge Join Operator [MERGEJOIN_1004] (rows=20714426 width=334)
+ Conds:RS_66._col4=RS_1089._col0(Inner),Output:["_col3","_col5","_col8","_col9","_col10","_col11","_col15","_col16","_col17","_col18","_col19","_col24","_col25"]
<-Map 46 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1039]
+ SHUFFLE [RS_1089]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1038]
+ Please refer to the previous Select Operator [SEL_1088]
<-Reducer 7 [SIMPLE_EDGE]
SHUFFLE [RS_66]
PartitionCols:_col4
- Merge Join Operator [MERGEJOIN_953] (rows=20714426 width=336)
- Conds:RS_63._col6=RS_1035._col0(Inner),Output:["_col3","_col4","_col5","_col8","_col9","_col10","_col11","_col15","_col16","_col17","_col18","_col19","_col24","_col25"]
+ Merge Join Operator [MERGEJOIN_1003] (rows=20714426 width=336)
+ Conds:RS_63._col6=RS_1085._col0(Inner),Output:["_col3","_col4","_col5","_col8","_col9","_col10","_col11","_col15","_col16","_col17","_col18","_col19","_col24","_col25"]
<-Map 45 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1035]
+ SHUFFLE [RS_1085]
PartitionCols:_col0
- Select Operator [SEL_1034] (rows=1704 width=181)
+ Select Operator [SEL_1084] (rows=1704 width=181)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_1033] (rows=1704 width=181)
+ Filter Operator [FIL_1083] (rows=1704 width=181)
predicate:(s_store_name is not null and s_zip is not null)
TableScan [TS_27] (rows=1704 width=181)
default@store,store,Tbl:COMPLETE,Col:COMPLETE,Output:["s_store_sk","s_store_name","s_zip"]
<-Reducer 6 [SIMPLE_EDGE]
SHUFFLE [RS_63]
PartitionCols:_col6
- Merge Join Operator [MERGEJOIN_952] (rows=20714426 width=160)
- Conds:RS_60._col1, _col7=RS_1031._col0, _col1(Inner),Output:["_col3","_col4","_col5","_col6","_col8","_col9","_col10","_col11","_col15","_col16","_col17","_col18","_col19"]
+ Merge Join Operator [MERGEJOIN_1002] (rows=20714426 width=160)
+ Conds:RS_60._col1, _col7=RS_1081._col0, _col1(Inner),Output:["_col3","_col4","_col5","_col6","_col8","_col9","_col10","_col11","_col15","_col16","_col17","_col18","_col19"]
<-Map 44 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1031]
+ SHUFFLE [RS_1081]
PartitionCols:_col0, _col1
- Select Operator [SEL_1030] (rows=57591150 width=8)
+ Select Operator [SEL_1080] (rows=57591150 width=8)
Output:["_col0","_col1"]
TableScan [TS_25] (rows=57591150 width=8)
default@store_returns,store_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["sr_item_sk","sr_ticket_number"]
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_60]
PartitionCols:_col1, _col7
- Merge Join Operator [MERGEJOIN_951] (rows=12564038 width=28)
- Conds:RS_57._col1=RS_1029._col0(Inner),Output:["_col1","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col15","_col16","_col17","_col18","_col19"]
+ Merge Join Operator [MERGEJOIN_1001] (rows=12564038 width=28)
+ Conds:RS_57._col1=RS_1079._col0(Inner),Output:["_col1","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col15","_col16","_col17","_col18","_col19"]
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_57]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_949] (rows=12564038 width=28)
- Conds:RS_54._col2=RS_1016._col0(Inner),Output:["_col1","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col15","_col16","_col17","_col18","_col19"]
+ Merge Join Operator [MERGEJOIN_999] (rows=12564038 width=28)
+ Conds:RS_54._col2=RS_1066._col0(Inner),Output:["_col1","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col15","_col16","_col17","_col18","_col19"]
<-Map 37 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1016]
+ SHUFFLE [RS_1066]
PartitionCols:_col0
- Select Operator [SEL_1015] (rows=69376329 width=23)
+ Select Operator [SEL_1065] (rows=69376329 width=23)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_1014] (rows=69376329 width=23)
+ Filter Operator [FIL_1064] (rows=69376329 width=23)
predicate:(c_first_shipto_date_sk is not null and c_first_sales_date_sk is not null and c_current_hdemo_sk is not null and c_current_cdemo_sk is not null and c_current_addr_sk is not null)
TableScan [TS_9] (rows=80000000 width=23)
default@customer,customer,Tbl:COMPLETE,Col:COMPLETE,Output:["c_customer_sk","c_current_cdemo_sk","c_current_hdemo_sk","c_current_addr_sk","c_first_shipto_date_sk","c_first_sales_date_sk"]
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_54]
PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_948] (rows=14487982 width=12)
- Conds:RS_51._col0=RS_990._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
+ Merge Join Operator [MERGEJOIN_998] (rows=14487982 width=12)
+ Conds:RS_51._col0=RS_1040._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
<-Map 47 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_990]
+ PARTITION_ONLY_SHUFFLE [RS_1040]
PartitionCols:_col0
- Select Operator [SEL_985] (rows=652 width=4)
+ Select Operator [SEL_1035] (rows=652 width=4)
Output:["_col0"]
- Filter Operator [FIL_979] (rows=652 width=8)
+ Filter Operator [FIL_1029] (rows=652 width=8)
predicate:(d_year = 2001)
Please refer to the previous TableScan [TS_38]
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_51]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_947] (rows=40575792 width=205)
- Conds:RS_1002._col1=RS_1006._col0(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
+ Merge Join Operator [MERGEJOIN_997] (rows=40575792 width=205)
+ Conds:RS_1052._col1=RS_1056._col0(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
<-Map 19 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1006]
+ PARTITION_ONLY_SHUFFLE [RS_1056]
PartitionCols:_col0
- Select Operator [SEL_1004] (rows=4667 width=4)
+ Select Operator [SEL_1054] (rows=4667 width=4)
Output:["_col0"]
- Filter Operator [FIL_1003] (rows=4667 width=204)
+ Filter Operator [FIL_1053] (rows=4667 width=204)
predicate:(i_current_price BETWEEN 36 AND 45 and (i_color) IN ('maroon', 'burnished', 'dim', 'steel', 'navajo', 'chocolate'))
TableScan [TS_3] (rows=462000 width=204)
default@item,item,Tbl:COMPLETE,Col:COMPLETE,Output:["i_item_sk","i_current_price","i_color","i_product_name"]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1002]
+ SHUFFLE [RS_1052]
PartitionCols:_col1
- Select Operator [SEL_1001] (rows=417313408 width=351)
+ Select Operator [SEL_1051] (rows=417313408 width=351)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10"]
- Filter Operator [FIL_1000] (rows=417313408 width=355)
+ Filter Operator [FIL_1050] (rows=417313408 width=355)
predicate:(ss_cdemo_sk is not null and ss_sold_date_sk is not null and ss_promo_sk is not null and ss_addr_sk is not null and ss_customer_sk is not null and ss_hdemo_sk is not null and ss_store_sk is not null and ss_sold_date_sk BETWEEN DynamicValue(RS_52_d1_d_date_sk_min) AND DynamicValue(RS_52_d1_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_52_d1_d_date_sk_bloom_filter)))
TableScan [TS_0] (rows=575995635 width=355)
default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_item_sk","ss_customer_sk","ss_cdemo_sk","ss_hdemo_sk","ss_addr_sk","ss_store_sk","ss_promo_sk","ss_ticket_number","ss_wholesale_cost","ss_list_price","ss_coupon_amt"]
<-Reducer 48 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_999]
- Group By Operator [GBY_998] (rows=1 width=12)
+ BROADCAST [RS_1049]
+ Group By Operator [GBY_1048] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 47 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_996]
- Group By Operator [GBY_994] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_1046]
+ Group By Operator [GBY_1044] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_991] (rows=652 width=4)
+ Select Operator [SEL_1041] (rows=652 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_985]
+ Please refer to the previous Select Operator [SEL_1035]
<-Reducer 40 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1029]
+ SHUFFLE [RS_1079]
PartitionCols:_col0
- Select Operator [SEL_1028] (rows=13257 width=4)
+ Select Operator [SEL_1078] (rows=13257 width=4)
Output:["_col0"]
- Filter Operator [FIL_1027] (rows=13257 width=228)
+ Filter Operator [FIL_1077] (rows=13257 width=228)
predicate:(_col1 > (2 * _col2))
- Group By Operator [GBY_1026] (rows=39773 width=228)
+ Group By Operator [GBY_1076] (rows=39773 width=228)
Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0
<-Reducer 39 [SIMPLE_EDGE]
SHUFFLE [RS_21]
PartitionCols:_col0
Group By Operator [GBY_20] (rows=6482999 width=228)
Output:["_col0","_col1","_col2"],aggregations:["sum(_col2)","sum(_col5)"],keys:_col0
- Merge Join Operator [MERGEJOIN_950] (rows=183085709 width=227)
- Conds:RS_1022._col0, _col1=RS_1024._col0, _col1(Inner),Output:["_col0","_col2","_col5"]
+ Merge Join Operator [MERGEJOIN_1000] (rows=183085709 width=227)
+ Conds:RS_1072._col0, _col1=RS_1074._col0, _col1(Inner),Output:["_col0","_col2","_col5"]
<-Map 41 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1024]
+ SHUFFLE [RS_1074]
PartitionCols:_col0, _col1
- Select Operator [SEL_1023] (rows=28798881 width=120)
+ Select Operator [SEL_1073] (rows=28798881 width=120)
Output:["_col0","_col1","_col2"]
TableScan [TS_14] (rows=28798881 width=337)
default@catalog_returns,catalog_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["cr_item_sk","cr_order_number","cr_refunded_cash","cr_reversed_charge","cr_store_credit"]
<-Map 38 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1022]
+ SHUFFLE [RS_1072]
PartitionCols:_col0, _col1
- Select Operator [SEL_1021] (rows=287989836 width=119)
+ Select Operator [SEL_1071] (rows=287989836 width=119)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_1020] (rows=287989836 width=119)
+ Filter Operator [FIL_1070] (rows=287989836 width=119)
predicate:(cs_item_sk BETWEEN DynamicValue(RS_49_item_i_item_sk_min) AND DynamicValue(RS_49_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_49_item_i_item_sk_bloom_filter)))
TableScan [TS_12] (rows=287989836 width=119)
default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["cs_item_sk","cs_order_number","cs_ext_list_price"]
<-Reducer 20 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_1019]
- Group By Operator [GBY_1018] (rows=1 width=12)
+ BROADCAST [RS_1069]
+ Group By Operator [GBY_1068] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 19 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1012]
- Group By Operator [GBY_1010] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_1062]
+ Group By Operator [GBY_1060] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_1007] (rows=4667 width=4)
+ Select Operator [SEL_1057] (rows=4667 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_1004]
+ Please refer to the previous Select Operator [SEL_1054]
<-Reducer 35 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1076]
+ SHUFFLE [RS_1126]
PartitionCols:_col2, _col1, _col3
- Select Operator [SEL_1075] (rows=21304422 width=1354)
+ Select Operator [SEL_1125] (rows=21304422 width=1354)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- Filter Operator [FIL_1074] (rows=21304422 width=1362)
+ Filter Operator [FIL_1124] (rows=21304422 width=1362)
predicate:_col14 is not null
- Select Operator [SEL_1073] (rows=21304422 width=1362)
+ Select Operator [SEL_1123] (rows=21304422 width=1362)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col14","_col15","_col16","_col17"]
- Group By Operator [GBY_1072] (rows=21304422 width=1362)
+ Group By Operator [GBY_1122] (rows=21304422 width=1362)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17"],aggregations:["count(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5, KEY._col6, KEY._col7, KEY._col8, KEY._col9, KEY._col10, KEY._col11, KEY._col12, KEY._col13
<-Reducer 34 [SIMPLE_EDGE]
SHUFFLE [RS_191]
PartitionCols:_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13
Group By Operator [GBY_190] (rows=21304422 width=1362)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17"],aggregations:["count()","sum(_col8)","sum(_col9)","sum(_col10)"],keys:_col24, _col11, _col25, _col12, _col29, _col31, _col37, _col38, _col39, _col40, _col42, _col43, _col44, _col45
- Merge Join Operator [MERGEJOIN_976] (rows=21304422 width=1155)
- Conds:RS_186._col17=RS_1052._col0(Inner),Output:["_col8","_col9","_col10","_col11","_col12","_col24","_col25","_col29","_col31","_col37","_col38","_col39","_col40","_col42","_col43","_col44","_col45"]
+ Merge Join Operator [MERGEJOIN_1026] (rows=21304422 width=1155)
+ Conds:RS_186._col17=RS_1102._col0(Inner),Output:["_col8","_col9","_col10","_col11","_col12","_col24","_col25","_col29","_col31","_col37","_col38","_col39","_col40","_col42","_col43","_col44","_col45"]
<-Map 51 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1052]
+ SHUFFLE [RS_1102]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1048]
+ Please refer to the previous Select Operator [SEL_1098]
<-Reducer 33 [SIMPLE_EDGE]
SHUFFLE [RS_186]
PartitionCols:_col17
- Merge Join Operator [MERGEJOIN_975] (rows=21304422 width=798)
- Conds:RS_183._col5=RS_1051._col0(Inner),Output:["_col8","_col9","_col10","_col11","_col12","_col17","_col24","_col25","_col29","_col31","_col37","_col38","_col39","_col40"]
+ Merge Join Operator [MERGEJOIN_1025] (rows=21304422 width=798)
+ Conds:RS_183._col5=RS_1101._col0(Inner),Output:["_col8","_col9","_col10","_col11","_col12","_col17","_col24","_col25","_col29","_col31","_col37","_col38","_col39","_col40"]
<-Map 51 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1051]
+ SHUFFLE [RS_1101]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1048]
+ Please refer to the previous Select Operator [SEL_1098]
<-Reducer 32 [SIMPLE_EDGE]
SHUFFLE [RS_183]
PartitionCols:_col5
Filter Operator [FIL_182] (rows=21304422 width=609)
predicate:(_col33 <> _col35)
- Merge Join Operator [MERGEJOIN_974] (rows=21304422 width=609)
- Conds:RS_179._col15=RS_1047._col0(Inner),Output:["_col5","_col8","_col9","_col10","_col11","_col12","_col17","_col24","_col25","_col29","_col31","_col33","_col35"]
+ Merge Join Operator [MERGEJOIN_1024] (rows=21304422 width=609)
+ Conds:RS_179._col15=RS_1097._col0(Inner),Output:["_col5","_col8","_col9","_col10","_col11","_col12","_col17","_col24","_col25","_col29","_col31","_col33","_col35"]
<-Map 50 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1047]
+ SHUFFLE [RS_1097]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1043]
+ Please refer to the previous Select Operator [SEL_1093]
<-Reducer 31 [SIMPLE_EDGE]
SHUFFLE [RS_179]
PartitionCols:_col15
- Merge Join Operator [MERGEJOIN_973] (rows=21007353 width=525)
- Conds:RS_176._col3=RS_1046._col0(Inner),Output:["_col5","_col8","_col9","_col10","_col11","_col12","_col15","_col17","_col24","_col25","_col29","_col31","_col33"]
+ Merge Join Operator [MERGEJOIN_1023] (rows=21007353 width=525)
+ Conds:RS_176._col3=RS_1096._col0(Inner),Output:["_col5","_col8","_col9","_col10","_col11","_col12","_col15","_col17","_col24","_col25","_col29","_col31","_col33"]
<-Map 50 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1046]
+ SHUFFLE [RS_1096]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1043]
+ Please refer to the previous Select Operator [SEL_1093]
<-Reducer 30 [SIMPLE_EDGE]
SHUFFLE [RS_176]
PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_972] (rows=20714426 width=438)
- Conds:RS_173._col18=RS_988._col0(Inner),Output:["_col3","_col5","_col8","_col9","_col10","_col11","_col12","_col15","_col17","_col24","_col25","_col29","_col31"]
+ Merge Join Operator [MERGEJOIN_1022] (rows=20714426 width=438)
+ Conds:RS_173._col18=RS_1038._col0(Inner),Output:["_col3","_col5","_col8","_col9","_col10","_col11","_col12","_col15","_col17","_col24","_col25","_col29","_col31"]
<-Map 47 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_988]
+ PARTITION_ONLY_SHUFFLE [RS_1038]
PartitionCols:_col0
- Select Operator [SEL_982] (rows=73049 width=8)
+ Select Operator [SEL_1032] (rows=73049 width=8)
Output:["_col0","_col1"]
Please refer to the previous TableScan [TS_38]
<-Reducer 29 [SIMPLE_EDGE]
SHUFFLE [RS_173]
PartitionCols:_col18
- Merge Join Operator [MERGEJOIN_971] (rows=20714426 width=438)
- Conds:RS_170._col19=RS_987._col0(Inner),Output:["_col3","_col5","_col8","_col9","_col10","_col11","_col12","_col15","_col17","_col18","_col24","_col25","_col29"]
+ Merge Join Operator [MERGEJOIN_1021] (rows=20714426 width=438)
+ Conds:RS_170._col19=RS_1037._col0(Inner),Output:["_col3","_col5","_col8","_col9","_col10","_col11","_col12","_col15","_col17","_col18","_col24","_col25","_col29"]
<-Map 47 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_987]
+ PARTITION_ONLY_SHUFFLE [RS_1037]
PartitionCols:_col0
- Select Operator [SEL_981] (rows=73049 width=8)
+ Select Operator [SEL_1031] (rows=73049 width=8)
Output:["_col0","_col1"]
Please refer to the previous TableScan [TS_38]
<-Reducer 28 [SIMPLE_EDGE]
SHUFFLE [RS_170]
PartitionCols:_col19
- Merge Join Operator [MERGEJOIN_970] (rows=20714426 width=437)
- Conds:RS_167._col16=RS_1042._col0(Inner),Output:["_col3","_col5","_col8","_col9","_col10","_col11","_col12","_col15","_col17","_col18","_col19","_col24","_col25"]
+ Merge Join Operator [MERGEJOIN_1020] (rows=20714426 width=437)
+ Conds:RS_167._col16=RS_1092._col0(Inner),Output:["_col3","_col5","_col8","_col9","_col10","_col11","_col12","_col15","_col17","_col18","_col19","_col24","_col25"]
<-Map 46 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1042]
+ SHUFFLE [RS_1092]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1038]
+ Please refer to the previous Select Operator [SEL_1088]
<-Reducer 27 [SIMPLE_EDGE]
SHUFFLE [RS_167]
PartitionCols:_col16
- Merge Join Operator [MERGEJOIN_969] (rows=20714426 width=441)
- Conds:RS_164._col4=RS_1041._col0(Inner),Output:["_col3","_col5","_col8","_col9","_col10","_col11","_col12","_col15","_col16","_col17","_col18","_col19","_col24","_col25"]
+ Merge Join Operator [MERGEJOIN_1019] (rows=20714426 width=441)
+ Conds:RS_164._col4=RS_1091._col0(Inner),Output:["_col3","_col5","_col8","_col9","_col10","_col11","_col12","_col15","_col16","_col17","_col18","_col19","_col24","_col25"]
<-Map 46 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1041]
+ SHUFFLE [RS_1091]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1038]
+ Please refer to the previous Select Operator [SEL_1088]
<-Reducer 26 [SIMPLE_EDGE]
SHUFFLE [RS_164]
PartitionCols:_col4
- Merge Join Operator [MERGEJOIN_968] (rows=20714426 width=443)
- Conds:RS_161._col6=RS_1036._col0(Inner),Output:["_col3","_col4","_col5","_col8","_col9","_col10","_col11","_col12","_col15","_col16","_col17","_col18","_col19","_col24","_col25"]
+ Merge Join Operator [MERGEJOIN_1018] (rows=20714426 width=443)
+ Conds:RS_161._col6=RS_1086._col0(Inner),Output:["_col3","_col4","_col5","_col8","_col9","_col10","_col11","_col12","_col15","_col16","_col17","_col18","_col19","_col24","_col25"]
<-Map 45 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1036]
+ SHUFFLE [RS_1086]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1034]
+ Please refer to the previous Select Operator [SEL_1084]
<-Reducer 25 [SIMPLE_EDGE]
SHUFFLE [RS_161]
PartitionCols:_col6
- Merge Join Operator [MERGEJOIN_967] (rows=20714426 width=267)
- Conds:RS_158._col1, _col7=RS_1032._col0, _col1(Inner),Output:["_col3","_col4","_col5","_col6","_col8","_col9","_col10","_col11","_col12","_col15","_col16","_col17","_col18","_col19"]
+ Merge Join Operator [MERGEJOIN_1017] (rows=20714426 width=267)
+ Conds:RS_158._col1, _col7=RS_1082._col0, _col1(Inner),Output:["_col3","_col4","_col5","_col6","_col8","_col9","_col10","_col11","_col12","_col15","_col16","_col17","_col18","_col19"]
<-Map 44 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1032]
+ SHUFFLE [RS_1082]
PartitionCols:_col0, _col1
- Please refer to the previous Select Operator [SEL_1030]
+ Please refer to the previous Select Operator [SEL_1080]
<-Reducer 24 [SIMPLE_EDGE]
SHUFFLE [RS_158]
PartitionCols:_col1, _col7
- Merge Join Operator [MERGEJOIN_966] (rows=12564038 width=135)
- Conds:RS_155._col1=RS_1071._col0(Inner),Output:["_col1","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col15","_col16","_col17","_col18","_col19"]
+ Merge Join Operator [MERGEJOIN_1016] (rows=12564038 width=135)
+ Conds:RS_155._col1=RS_1121._col0(Inner),Output:["_col1","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col15","_col16","_col17","_col18","_col19"]
<-Reducer 23 [SIMPLE_EDGE]
SHUFFLE [RS_155]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_964] (rows=12564038 width=135)
- Conds:RS_152._col2=RS_1017._col0(Inner),Output:["_col1","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col15","_col16","_col17","_col18","_col19"]
+ Merge Join Operator [MERGEJOIN_1014] (rows=12564038 width=135)
+ Conds:RS_152._col2=RS_1067._col0(Inner),Output:["_col1","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col15","_col16","_col17","_col18","_col19"]
<-Map 37 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1017]
+ SHUFFLE [RS_1067]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1015]
+ Please refer to the previous Select Operator [SEL_1065]
<-Reducer 22 [SIMPLE_EDGE]
SHUFFLE [RS_152]
PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_963] (rows=14487982 width=119)
- Conds:RS_149._col0=RS_992._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12"]
+ Merge Join Operator [MERGEJOIN_1013] (rows=14487982 width=119)
+ Conds:RS_149._col0=RS_1042._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12"]
<-Map 47 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_992]
+ PARTITION_ONLY_SHUFFLE [RS_1042]
PartitionCols:_col0
- Select Operator [SEL_989] (rows=652 width=4)
+ Select Operator [SEL_1039] (rows=652 width=4)
Output:["_col0"]
- Filter Operator [FIL_983] (rows=652 width=8)
+ Filter Operator [FIL_1033] (rows=652 width=8)
predicate:(d_year = 2000)
Please refer to the previous TableScan [TS_38]
<-Reducer 21 [SIMPLE_EDGE]
SHUFFLE [RS_149]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_962] (rows=40575792 width=312)
- Conds:RS_1062._col1=RS_1008._col0(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12"]
+ Merge Join Operator [MERGEJOIN_1012] (rows=40575792 width=312)
+ Conds:RS_1112._col1=RS_1058._col0(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12"]
<-Map 19 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1008]
+ PARTITION_ONLY_SHUFFLE [RS_1058]
PartitionCols:_col0
- Select Operator [SEL_1005] (rows=4667 width=111)
+ Select Operator [SEL_1055] (rows=4667 width=111)
Output:["_col0","_col1"]
- Please refer to the previous Filter Operator [FIL_1003]
+ Please refer to the previous Filter Operator [FIL_1053]
<-Map 52 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1062]
+ SHUFFLE [RS_1112]
PartitionCols:_col1
- Select Operator [SEL_1061] (rows=417313408 width=351)
+ Select Operator [SEL_1111] (rows=417313408 width=351)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10"]
- Filter Operator [FIL_1060] (rows=417313408 width=355)
+ Filter Operator [FIL_1110] (rows=417313408 width=355)
predicate:(ss_cdemo_sk is not null and ss_sold_date_sk is not null and ss_promo_sk is not null and ss_addr_sk is not null and ss_customer_sk is not null and ss_hdemo_sk is not null and ss_store_sk is not null and ss_sold_date_sk BETWEEN DynamicValue(RS_150_d1_d_date_sk_min) AND DynamicValue(RS_150_d1_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_150_d1_d_date_sk_bloom_filter)))
TableScan [TS_98] (rows=575995635 width=355)
default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_item_sk","ss_customer_sk","ss_cdemo_sk","ss_hdemo_sk","ss_addr_sk","ss_store_sk","ss_promo_sk","ss_ticket_number","ss_wholesale_cost","ss_list_price","ss_coupon_amt"]
<-Reducer 49 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_1059]
- Group By Operator [GBY_1058] (rows=1 width=12)
+ BROADCAST [RS_1109]
+ Group By Operator [GBY_1108] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 47 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_997]
- Group By Operator [GBY_995] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_1047]
+ Group By Operator [GBY_1045] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_993] (rows=652 width=4)
+ Select Operator [SEL_1043] (rows=652 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_989]
+ Please refer to the previous Select Operator [SEL_1039]
<-Reducer 43 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1071]
+ SHUFFLE [RS_1121]
PartitionCols:_col0
- Select Operator [SEL_1070] (rows=13257 width=4)
+ Select Operator [SEL_1120] (rows=13257 width=4)
Output:["_col0"]
- Filter Operator [FIL_1069] (rows=13257 width=228)
+ Filter Operator [FIL_1119] (rows=13257 width=228)
predicate:(_col1 > (2 * _col2))
- Group By Operator [GBY_1068] (rows=39773 width=228)
+ Group By Operator [GBY_1118] (rows=39773 width=228)
Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0
<-Reducer 42 [SIMPLE_EDGE]
SHUFFLE [RS_119]
PartitionCols:_col0
Group By Operator [GBY_118] (rows=6482999 width=228)
Output:["_col0","_col1","_col2"],aggregations:["sum(_col2)","sum(_col5)"],keys:_col0
- Merge Join Operator [MERGEJOIN_965] (rows=183085709 width=227)
- Conds:RS_1067._col0, _col1=RS_1025._col0, _col1(Inner),Output:["_col0","_col2","_col5"]
+ Merge Join Operator [MERGEJOIN_1015] (rows=183085709 width=227)
+ Conds:RS_1117._col0, _col1=RS_1075._col0, _col1(Inner),Output:["_col0","_col2","_col5"]
<-Map 41 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1025]
+ SHUFFLE [RS_1075]
PartitionCols:_col0, _col1
- Please refer to the previous Select Operator [SEL_1023]
+ Please refer to the previous Select Operator [SEL_1073]
<-Map 53 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1067]
+ SHUFFLE [RS_1117]
PartitionCols:_col0, _col1
- Select Operator [SEL_1066] (rows=287989836 width=119)
+ Select Operator [SEL_1116] (rows=287989836 width=119)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_1065] (rows=287989836 width=119)
+ Filter Operator [FIL_1115] (rows=287989836 width=119)
predicate:(cs_item_sk BETWEEN DynamicValue(RS_147_item_i_item_sk_min) AND DynamicValue(RS_147_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_147_item_i_item_sk_bloom_filter)))
TableScan [TS_110] (rows=287989836 width=119)
default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["cs_item_sk","cs_order_number","cs_ext_list_price"]
<-Reducer 36 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_1064]
- Group By Operator [GBY_1063] (rows=1 width=12)
+ BROADCAST [RS_1114]
+ Group By Operator [GBY_1113] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 19 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1013]
- Group By Operator [GBY_1011] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_1063]
+ Group By Operator [GBY_1061] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_1009] (rows=4667 width=4)
+ Select Operator [SEL_1059] (rows=4667 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_1005]
+ Please refer to the previous Select Operator [SEL_1055]
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query72.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query72.q.out
index 1308e79..dc4e50a 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query72.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query72.q.out
@@ -101,16 +101,16 @@ Stage-0
limit:100
Stage-1
Reducer 12 vectorized
- File Output Operator [FS_292]
- Limit [LIM_291] (rows=100 width=312)
+ File Output Operator [FS_302]
+ Limit [LIM_301] (rows=100 width=312)
Number of rows:100
- Select Operator [SEL_290] (rows=471834849 width=312)
+ Select Operator [SEL_300] (rows=471834849 width=312)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
<-Reducer 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_289]
- Top N Key Operator [TNK_288] (rows=471834849 width=312)
+ SHUFFLE [RS_299]
+ Top N Key Operator [TNK_298] (rows=471834849 width=312)
keys:_col5, _col0, _col1, _col2,top n:100
- Group By Operator [GBY_287] (rows=471834849 width=312)
+ Group By Operator [GBY_297] (rows=471834849 width=312)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["count(VALUE._col0)","count(VALUE._col1)","count(VALUE._col2)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 10 [SIMPLE_EDGE]
SHUFFLE [RS_66]
@@ -119,36 +119,36 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["count(_col3)","count(_col4)","count()"],keys:_col0, _col1, _col2
Select Operator [SEL_63] (rows=471834849 width=292)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Merge Join Operator [MERGEJOIN_255] (rows=471834849 width=292)
- Conds:RS_60._col4=RS_286._col0(Inner),Output:["_col15","_col21","_col25","_col27"]
+ Merge Join Operator [MERGEJOIN_265] (rows=471834849 width=292)
+ Conds:RS_60._col4=RS_296._col0(Inner),Output:["_col15","_col21","_col25","_col27"]
<-Map 23 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_286]
+ SHUFFLE [RS_296]
PartitionCols:_col0
- Select Operator [SEL_285] (rows=462000 width=188)
+ Select Operator [SEL_295] (rows=462000 width=188)
Output:["_col0","_col1"]
TableScan [TS_52] (rows=462000 width=188)
default@item,item,Tbl:COMPLETE,Col:COMPLETE,Output:["i_item_sk","i_item_desc"]
<-Reducer 9 [SIMPLE_EDGE]
SHUFFLE [RS_60]
PartitionCols:_col4
- Merge Join Operator [MERGEJOIN_254] (rows=471834849 width=112)
- Conds:RS_57._col10=RS_284._col0(Inner),Output:["_col4","_col15","_col21","_col25"]
+ Merge Join Operator [MERGEJOIN_264] (rows=471834849 width=112)
+ Conds:RS_57._col10=RS_294._col0(Inner),Output:["_col4","_col15","_col21","_col25"]
<-Map 22 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_284]
+ SHUFFLE [RS_294]
PartitionCols:_col0
- Select Operator [SEL_283] (rows=27 width=104)
+ Select Operator [SEL_293] (rows=27 width=104)
Output:["_col0","_col1"]
TableScan [TS_50] (rows=27 width=104)
default@warehouse,warehouse,Tbl:COMPLETE,Col:COMPLETE,Output:["w_warehouse_sk","w_warehouse_name"]
<-Reducer 8 [SIMPLE_EDGE]
SHUFFLE [RS_57]
PartitionCols:_col10
- Merge Join Operator [MERGEJOIN_253] (rows=471834849 width=16)
- Conds:RS_54._col4, _col6=RS_282._col0, _col1(Left Outer),Output:["_col4","_col10","_col15","_col21"]
+ Merge Join Operator [MERGEJOIN_263] (rows=471834849 width=16)
+ Conds:RS_54._col4, _col6=RS_292._col0, _col1(Left Outer),Output:["_col4","_col10","_col15","_col21"]
<-Map 21 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_282]
+ SHUFFLE [RS_292]
PartitionCols:_col0, _col1
- Select Operator [SEL_281] (rows=28798881 width=8)
+ Select Operator [SEL_291] (rows=28798881 width=8)
Output:["_col0","_col1"]
TableScan [TS_48] (rows=28798881 width=8)
default@catalog_returns,catalog_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["cr_item_sk","cr_order_number"]
@@ -159,26 +159,26 @@ Stage-0
Output:["_col4","_col6","_col10","_col15","_col21"]
Filter Operator [FIL_46] (rows=182953402 width=36)
predicate:(_col21 > _col14)
- Merge Join Operator [MERGEJOIN_252] (rows=548860207 width=36)
- Conds:RS_43._col1=RS_261._col0(Inner),Output:["_col4","_col6","_col13","_col14","_col17","_col19","_col21"]
+ Merge Join Operator [MERGEJOIN_262] (rows=548860207 width=36)
+ Conds:RS_43._col1=RS_271._col0(Inner),Output:["_col4","_col6","_col13","_col14","_col17","_col19","_col21"]
<-Map 15 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_261]
+ SHUFFLE [RS_271]
PartitionCols:_col0
- Select Operator [SEL_259] (rows=73049 width=12)
+ Select Operator [SEL_269] (rows=73049 width=12)
Output:["_col0","_col1"]
- Filter Operator [FIL_257] (rows=73049 width=98)
+ Filter Operator [FIL_267] (rows=73049 width=98)
predicate:UDFToDouble(d_date) is not null
TableScan [TS_9] (rows=73049 width=8)
default@date_dim,d2,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_week_seq","d_date"]
<-Reducer 6 [SIMPLE_EDGE]
SHUFFLE [RS_43]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_251] (rows=548860207 width=31)
- Conds:RS_40._col5=RS_280._col0(Left Outer),Output:["_col1","_col4","_col6","_col13","_col14","_col17","_col19"]
+ Merge Join Operator [MERGEJOIN_261] (rows=548860207 width=31)
+ Conds:RS_40._col5=RS_290._col0(Left Outer),Output:["_col1","_col4","_col6","_col13","_col14","_col17","_col19"]
<-Map 20 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_280]
+ SHUFFLE [RS_290]
PartitionCols:_col0
- Select Operator [SEL_279] (rows=2300 width=4)
+ Select Operator [SEL_289] (rows=2300 width=4)
Output:["_col0"]
TableScan [TS_22] (rows=2300 width=4)
default@promotion,promotion,Tbl:COMPLETE,Col:COMPLETE,Output:["p_promo_sk"]
@@ -187,75 +187,75 @@ Stage-0
PartitionCols:_col5
Filter Operator [FIL_39] (rows=548860207 width=39)
predicate:(_col18 < _col7)
- Merge Join Operator [MERGEJOIN_250] (rows=1646580622 width=39)
- Conds:RS_36._col10, _col4=RS_278._col0, _col1(Inner),Output:["_col1","_col4","_col5","_col6","_col7","_col13","_col14","_col17","_col18"]
+ Merge Join Operator [MERGEJOIN_260] (rows=1646580622 width=39)
+ Conds:RS_36._col10, _col4=RS_288._col0, _col1(Inner),Output:["_col1","_col4","_col5","_col6","_col7","_col13","_col14","_col17","_col18"]
<-Map 19 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_278]
+ SHUFFLE [RS_288]
PartitionCols:_col0, _col1
- Select Operator [SEL_277] (rows=35703276 width=15)
+ Select Operator [SEL_287] (rows=35703276 width=15)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_276] (rows=35703276 width=15)
+ Filter Operator [FIL_286] (rows=35703276 width=15)
predicate:inv_quantity_on_hand is not null
TableScan [TS_19] (rows=37584000 width=15)
default@inventory,inventory,Tbl:COMPLETE,Col:COMPLETE,Output:["inv_date_sk","inv_item_sk","inv_warehouse_sk","inv_quantity_on_hand"]
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_36]
PartitionCols:_col10, _col4
- Merge Join Operator [MERGEJOIN_249] (rows=2885264 width=30)
+ Merge Join Operator [MERGEJOIN_259] (rows=2885264 width=30)
Conds:RS_33._col0=RS_34._col2(Inner),Output:["_col1","_col4","_col5","_col6","_col7","_col10","_col13","_col14"]
<-Reducer 16 [SIMPLE_EDGE]
SHUFFLE [RS_34]
PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_248] (rows=3621 width=20)
- Conds:RS_260._col1=RS_264._col1(Inner),Output:["_col0","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_258] (rows=3621 width=20)
+ Conds:RS_270._col1=RS_274._col1(Inner),Output:["_col0","_col2","_col3","_col4"]
<-Map 15 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_260]
+ SHUFFLE [RS_270]
PartitionCols:_col1
- Select Operator [SEL_258] (rows=73049 width=8)
+ Select Operator [SEL_268] (rows=73049 width=8)
Output:["_col0","_col1"]
- Filter Operator [FIL_256] (rows=73049 width=8)
+ Filter Operator [FIL_266] (rows=73049 width=8)
predicate:d_week_seq is not null
Please refer to the previous TableScan [TS_9]
<-Map 18 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_264]
+ SHUFFLE [RS_274]
PartitionCols:_col1
- Select Operator [SEL_263] (rows=652 width=16)
+ Select Operator [SEL_273] (rows=652 width=16)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_262] (rows=652 width=106)
+ Filter Operator [FIL_272] (rows=652 width=106)
predicate:((d_year = 2001) and d_week_seq is not null and UDFToDouble(d_date) is not null)
TableScan [TS_12] (rows=73049 width=106)
default@date_dim,d1,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_date","d_week_seq","d_year"]
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_33]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_247] (rows=8138146 width=21)
- Conds:RS_30._col3=RS_275._col0(Inner),Output:["_col0","_col1","_col4","_col5","_col6","_col7"]
+ Merge Join Operator [MERGEJOIN_257] (rows=8138146 width=21)
+ Conds:RS_30._col3=RS_285._col0(Inner),Output:["_col0","_col1","_col4","_col5","_col6","_col7"]
<-Map 14 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_275]
+ SHUFFLE [RS_285]
PartitionCols:_col0
- Select Operator [SEL_274] (rows=1440 width=4)
+ Select Operator [SEL_284] (rows=1440 width=4)
Output:["_col0"]
- Filter Operator [FIL_273] (rows=1440 width=96)
+ Filter Operator [FIL_283] (rows=1440 width=96)
predicate:(hd_buy_potential = '1001-5000')
TableScan [TS_6] (rows=7200 width=96)
default@household_demographics,household_demographics,Tbl:COMPLETE,Col:COMPLETE,Output:["hd_demo_sk","hd_buy_potential"]
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_30]
PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_246] (rows=40690727 width=27)
- Conds:RS_269._col2=RS_272._col0(Inner),Output:["_col0","_col1","_col3","_col4","_col5","_col6","_col7"]
+ Merge Join Operator [MERGEJOIN_256] (rows=40690727 width=27)
+ Conds:RS_279._col2=RS_282._col0(Inner),Output:["_col0","_col1","_col3","_col4","_col5","_col6","_col7"]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_269]
+ SHUFFLE [RS_279]
PartitionCols:_col2
- Select Operator [SEL_268] (rows=280863798 width=31)
+ Select Operator [SEL_278] (rows=280863798 width=31)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- Filter Operator [FIL_267] (rows=280863798 width=31)
+ Filter Operator [FIL_277] (rows=280863798 width=31)
predicate:(cs_sold_date_sk is not null and cs_bill_cdemo_sk is not null and cs_ship_date_sk is not null and cs_quantity is not null and cs_bill_hdemo_sk is not null and cs_sold_date_sk BETWEEN DynamicValue(RS_34_d1_d_date_sk_min) AND DynamicValue(RS_34_d1_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_34_d1_d_date_sk_bloom_filter)))
TableScan [TS_0] (rows=287989836 width=31)
default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["cs_sold_date_sk","cs_ship_date_sk","cs_bill_cdemo_sk","cs_bill_hdemo_sk","cs_item_sk","cs_promo_sk","cs_order_number","cs_quantity"]
<-Reducer 17 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_266]
- Group By Operator [GBY_265] (rows=1 width=12)
+ BROADCAST [RS_276]
+ Group By Operator [GBY_275] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Reducer 16 [CUSTOM_SIMPLE_EDGE]
SHUFFLE [RS_146]
@@ -263,13 +263,13 @@ Stage-0
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
Select Operator [SEL_144] (rows=3621 width=8)
Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_248]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_258]
<-Map 13 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_272]
+ SHUFFLE [RS_282]
PartitionCols:_col0
- Select Operator [SEL_271] (rows=265971 width=4)
+ Select Operator [SEL_281] (rows=265971 width=4)
Output:["_col0"]
- Filter Operator [FIL_270] (rows=265971 width=89)
+ Filter Operator [FIL_280] (rows=265971 width=89)
predicate:(cd_marital_status = 'M')
TableScan [TS_3] (rows=1861800 width=89)
default@customer_demographics,customer_demographics,Tbl:COMPLETE,Col:COMPLETE,Output:["cd_demo_sk","cd_marital_status"]
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query75.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query75.q.out
index a705dd8..782f8f9 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query75.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query75.q.out
@@ -199,12 +199,12 @@ Stage-0
limit:-1
Stage-1
Reducer 10 vectorized
- File Output Operator [FS_612]
- Select Operator [SEL_611] (rows=100 width=160)
+ File Output Operator [FS_672]
+ Select Operator [SEL_671] (rows=100 width=160)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
- Limit [LIM_610] (rows=100 width=152)
+ Limit [LIM_670] (rows=100 width=152)
Number of rows:100
- Select Operator [SEL_609] (rows=3422897230256 width=151)
+ Select Operator [SEL_669] (rows=3422897230256 width=151)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
<-Reducer 9 [SIMPLE_EDGE]
SHUFFLE [RS_169]
@@ -214,375 +214,375 @@ Stage-0
keys:(_col10 - _col4),top n:100
Filter Operator [FIL_167] (rows=3422897230256 width=255)
predicate:((CAST( _col10 AS decimal(17,2)) / CAST( _col4 AS decimal(17,2))) < 0.9)
- Merge Join Operator [MERGEJOIN_513] (rows=10268691690770 width=255)
- Conds:RS_605._col0, _col1, _col2, _col3=RS_608._col0, _col1, _col2, _col3(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col10","_col11"]
+ Merge Join Operator [MERGEJOIN_573] (rows=10268691690770 width=255)
+ Conds:RS_665._col0, _col1, _col2, _col3=RS_668._col0, _col1, _col2, _col3(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col10","_col11"]
<-Reducer 27 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_608]
+ SHUFFLE [RS_668]
PartitionCols:_col0, _col1, _col2, _col3
- Group By Operator [GBY_607] (rows=84235776 width=135)
+ Group By Operator [GBY_667] (rows=84235776 width=135)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col4)","sum(_col5)"],keys:_col0, _col1, _col2, _col3
- Group By Operator [GBY_606] (rows=736356923 width=131)
+ Group By Operator [GBY_666] (rows=736356923 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5
<-Union 26 [SIMPLE_EDGE]
<-Reducer 25 [CONTAINS] vectorized
- Reduce Output Operator [RS_636]
+ Reduce Output Operator [RS_696]
PartitionCols:_col0, _col1, _col2, _col3
- Group By Operator [GBY_635] (rows=736356923 width=131)
+ Group By Operator [GBY_695] (rows=736356923 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],keys:_col0, _col1, _col2, _col3, _col4, _col5
- Group By Operator [GBY_634] (rows=621178955 width=131)
+ Group By Operator [GBY_694] (rows=621178955 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5
<-Union 24 [SIMPLE_EDGE]
<-Reducer 23 [CONTAINS]
- Reduce Output Operator [RS_537]
+ Reduce Output Operator [RS_597]
PartitionCols:_col0, _col1, _col2, _col3, _col4, _col5
- Group By Operator [GBY_536] (rows=621178955 width=131)
+ Group By Operator [GBY_596] (rows=621178955 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],keys:_col0, _col1, _col2, _col3, _col4, _col5
- Select Operator [SEL_534] (rows=170474971 width=131)
+ Select Operator [SEL_594] (rows=170474971 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Merge Join Operator [MERGEJOIN_533] (rows=170474971 width=234)
- Conds:RS_99._col1, _col2=RS_599._col0, _col1(Left Outer),Output:["_col3","_col4","_col7","_col8","_col9","_col10","_col13","_col14"]
+ Merge Join Operator [MERGEJOIN_593] (rows=170474971 width=234)
+ Conds:RS_99._col1, _col2=RS_659._col0, _col1(Left Outer),Output:["_col3","_col4","_col7","_col8","_col9","_col10","_col13","_col14"]
<-Map 38 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_599]
+ SHUFFLE [RS_659]
PartitionCols:_col0, _col1
- Select Operator [SEL_597] (rows=28798881 width=121)
+ Select Operator [SEL_657] (rows=28798881 width=121)
Output:["_col0","_col1","_col2","_col3"]
TableScan [TS_9] (rows=28798881 width=121)
default@catalog_returns,catalog_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["cr_item_sk","cr_order_number","cr_return_quantity","cr_return_amount"]
<-Reducer 22 [SIMPLE_EDGE]
SHUFFLE [RS_99]
PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_505] (rows=96821196 width=138)
- Conds:RS_96._col1=RS_594._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col7","_col8","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_565] (rows=96821196 width=138)
+ Conds:RS_96._col1=RS_654._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col7","_col8","_col9","_col10"]
<-Map 37 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_594]
+ SHUFFLE [RS_654]
PartitionCols:_col0
- Select Operator [SEL_590] (rows=45745 width=19)
+ Select Operator [SEL_650] (rows=45745 width=19)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_589] (rows=45745 width=109)
+ Filter Operator [FIL_649] (rows=45745 width=109)
predicate:((i_category = 'Sports') and i_manufact_id is not null and i_category_id is not null and i_brand_id is not null and i_class_id is not null)
TableScan [TS_6] (rows=462000 width=109)
default@item,item,Tbl:COMPLETE,Col:COMPLETE,Output:["i_item_sk","i_brand_id","i_class_id","i_category_id","i_category","i_manufact_id"]
<-Reducer 21 [SIMPLE_EDGE]
SHUFFLE [RS_96]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_504] (rows=101592102 width=122)
- Conds:RS_633._col0=RS_566._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_564] (rows=101592102 width=122)
+ Conds:RS_693._col0=RS_626._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
<-Map 11 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_566]
+ PARTITION_ONLY_SHUFFLE [RS_626]
PartitionCols:_col0
- Select Operator [SEL_557] (rows=652 width=4)
+ Select Operator [SEL_617] (rows=652 width=4)
Output:["_col0"]
- Filter Operator [FIL_553] (rows=652 width=8)
+ Filter Operator [FIL_613] (rows=652 width=8)
predicate:(d_year = 2002)
TableScan [TS_3] (rows=73049 width=8)
default@date_dim,date_dim,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_year"]
<-Map 43 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_633]
+ SHUFFLE [RS_693]
PartitionCols:_col0
- Select Operator [SEL_632] (rows=286549727 width=127)
+ Select Operator [SEL_692] (rows=286549727 width=127)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_631] (rows=286549727 width=127)
+ Filter Operator [FIL_691] (rows=286549727 width=127)
predicate:(cs_sold_date_sk is not null and cs_sold_date_sk BETWEEN DynamicValue(RS_94_date_dim_d_date_sk_min) AND DynamicValue(RS_94_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_94_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_82] (rows=287989836 width=127)
default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["cs_sold_date_sk","cs_item_sk","cs_order_number","cs_quantity","cs_ext_sales_price"]
<-Reducer 28 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_630]
- Group By Operator [GBY_629] (rows=1 width=12)
+ BROADCAST [RS_690]
+ Group By Operator [GBY_689] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 11 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_581]
- Group By Operator [GBY_575] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_641]
+ Group By Operator [GBY_635] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_567] (rows=652 width=4)
+ Select Operator [SEL_627] (rows=652 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_557]
+ Please refer to the previous Select Operator [SEL_617]
<-Reducer 31 [CONTAINS]
- Reduce Output Operator [RS_546]
+ Reduce Output Operator [RS_606]
PartitionCols:_col0, _col1, _col2, _col3, _col4, _col5
- Group By Operator [GBY_545] (rows=621178955 width=131)
+ Group By Operator [GBY_605] (rows=621178955 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],keys:_col0, _col1, _col2, _col3, _col4, _col5
- Select Operator [SEL_543] (rows=450703984 width=131)
+ Select Operator [SEL_603] (rows=450703984 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Merge Join Operator [MERGEJOIN_542] (rows=450703984 width=204)
- Conds:RS_120._col1, _col2=RS_620._col0, _col1(Left Outer),Output:["_col3","_col4","_col7","_col8","_col9","_col10","_col13","_col14"]
+ Merge Join Operator [MERGEJOIN_602] (rows=450703984 width=204)
+ Conds:RS_120._col1, _col2=RS_680._col0, _col1(Left Outer),Output:["_col3","_col4","_col7","_col8","_col9","_col10","_col13","_col14"]
<-Map 40 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_620]
+ SHUFFLE [RS_680]
PartitionCols:_col0, _col1
- Select Operator [SEL_618] (rows=57591150 width=119)
+ Select Operator [SEL_678] (rows=57591150 width=119)
Output:["_col0","_col1","_col2","_col3"]
TableScan [TS_30] (rows=57591150 width=119)
default@store_returns,store_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["sr_item_sk","sr_ticket_number","sr_return_quantity","sr_return_amt"]
<-Reducer 30 [SIMPLE_EDGE]
SHUFFLE [RS_120]
PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_508] (rows=187186493 width=124)
- Conds:RS_117._col1=RS_595._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col7","_col8","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_568] (rows=187186493 width=124)
+ Conds:RS_117._col1=RS_655._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col7","_col8","_col9","_col10"]
<-Map 37 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_595]
+ SHUFFLE [RS_655]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_590]
+ Please refer to the previous Select Operator [SEL_650]
<-Reducer 29 [SIMPLE_EDGE]
SHUFFLE [RS_117]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_507] (rows=196410188 width=109)
- Conds:RS_641._col0=RS_568._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_567] (rows=196410188 width=109)
+ Conds:RS_701._col0=RS_628._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
<-Map 11 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_568]
+ PARTITION_ONLY_SHUFFLE [RS_628]
PartitionCols:_col0
- Select Operator [SEL_558] (rows=652 width=4)
+ Select Operator [SEL_618] (rows=652 width=4)
Output:["_col0"]
- Filter Operator [FIL_554] (rows=652 width=8)
+ Filter Operator [FIL_614] (rows=652 width=8)
predicate:(d_year = 2002)
Please refer to the previous TableScan [TS_3]
<-Map 44 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_641]
+ SHUFFLE [RS_701]
PartitionCols:_col0
- Select Operator [SEL_640] (rows=550076554 width=122)
+ Select Operator [SEL_700] (rows=550076554 width=122)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_639] (rows=550076554 width=122)
+ Filter Operator [FIL_699] (rows=550076554 width=122)
predicate:(ss_sold_date_sk is not null and ss_sold_date_sk BETWEEN DynamicValue(RS_115_date_dim_d_date_sk_min) AND DynamicValue(RS_115_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_115_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_103] (rows=575995635 width=122)
default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_item_sk","ss_ticket_number","ss_quantity","ss_ext_sales_price"]
<-Reducer 32 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_638]
- Group By Operator [GBY_637] (rows=1 width=12)
+ BROADCAST [RS_698]
+ Group By Operator [GBY_697] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 11 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_582]
- Group By Operator [GBY_576] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_642]
+ Group By Operator [GBY_636] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_569] (rows=652 width=4)
+ Select Operator [SEL_629] (rows=652 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_558]
+ Please refer to the previous Select Operator [SEL_618]
<-Reducer 35 [CONTAINS]
- Reduce Output Operator [RS_551]
+ Reduce Output Operator [RS_611]
PartitionCols:_col0, _col1, _col2, _col3
- Group By Operator [GBY_550] (rows=736356923 width=131)
+ Group By Operator [GBY_610] (rows=736356923 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],keys:_col0, _col1, _col2, _col3, _col4, _col5
- Select Operator [SEL_548] (rows=115177968 width=131)
+ Select Operator [SEL_608] (rows=115177968 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Merge Join Operator [MERGEJOIN_547] (rows=115177968 width=220)
- Conds:RS_148._col1, _col2=RS_628._col0, _col1(Left Outer),Output:["_col3","_col4","_col7","_col8","_col9","_col10","_col13","_col14"]
+ Merge Join Operator [MERGEJOIN_607] (rows=115177968 width=220)
+ Conds:RS_148._col1, _col2=RS_688._col0, _col1(Left Outer),Output:["_col3","_col4","_col7","_col8","_col9","_col10","_col13","_col14"]
<-Map 42 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_628]
+ SHUFFLE [RS_688]
PartitionCols:_col0, _col1
- Select Operator [SEL_626] (rows=14398467 width=118)
+ Select Operator [SEL_686] (rows=14398467 width=118)
Output:["_col0","_col1","_col2","_col3"]
TableScan [TS_58] (rows=14398467 width=118)
default@web_returns,web_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["wr_item_sk","wr_order_number","wr_return_quantity","wr_return_amt"]
<-Reducer 34 [SIMPLE_EDGE]
SHUFFLE [RS_148]
PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_511] (rows=48990732 width=139)
- Conds:RS_145._col1=RS_596._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col7","_col8","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_571] (rows=48990732 width=139)
+ Conds:RS_145._col1=RS_656._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col7","_col8","_col9","_col10"]
<-Map 37 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_596]
+ SHUFFLE [RS_656]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_590]
+ Please refer to the previous Select Operator [SEL_650]
<-Reducer 33 [SIMPLE_EDGE]
SHUFFLE [RS_145]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_510] (rows=51404771 width=123)
- Conds:RS_646._col0=RS_570._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_570] (rows=51404771 width=123)
+ Conds:RS_706._col0=RS_630._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
<-Map 11 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_570]
+ PARTITION_ONLY_SHUFFLE [RS_630]
PartitionCols:_col0
- Select Operator [SEL_559] (rows=652 width=4)
+ Select Operator [SEL_619] (rows=652 width=4)
Output:["_col0"]
- Filter Operator [FIL_555] (rows=652 width=8)
+ Filter Operator [FIL_615] (rows=652 width=8)
predicate:(d_year = 2002)
Please refer to the previous TableScan [TS_3]
<-Map 45 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_646]
+ SHUFFLE [RS_706]
PartitionCols:_col0
- Select Operator [SEL_645] (rows=143966864 width=127)
+ Select Operator [SEL_705] (rows=143966864 width=127)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_644] (rows=143966864 width=127)
+ Filter Operator [FIL_704] (rows=143966864 width=127)
predicate:(ws_sold_date_sk is not null and ws_sold_date_sk BETWEEN DynamicValue(RS_143_date_dim_d_date_sk_min) AND DynamicValue(RS_143_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_143_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_131] (rows=144002668 width=127)
default@web_sales,web_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ws_sold_date_sk","ws_item_sk","ws_order_number","ws_quantity","ws_ext_sales_price"]
<-Reducer 36 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_643]
- Group By Operator [GBY_642] (rows=1 width=12)
+ BROADCAST [RS_703]
+ Group By Operator [GBY_702] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 11 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_583]
- Group By Operator [GBY_577] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_643]
+ Group By Operator [GBY_637] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_571] (rows=652 width=4)
+ Select Operator [SEL_631] (rows=652 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_559]
+ Please refer to the previous Select Operator [SEL_619]
<-Reducer 8 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_605]
+ SHUFFLE [RS_665]
PartitionCols:_col0, _col1, _col2, _col3
- Group By Operator [GBY_604] (rows=84235776 width=135)
+ Group By Operator [GBY_664] (rows=84235776 width=135)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col4)","sum(_col5)"],keys:_col0, _col1, _col2, _col3
- Group By Operator [GBY_603] (rows=736356923 width=131)
+ Group By Operator [GBY_663] (rows=736356923 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5
<-Union 7 [SIMPLE_EDGE]
<-Reducer 19 [CONTAINS]
- Reduce Output Operator [RS_532]
+ Reduce Output Operator [RS_592]
PartitionCols:_col0, _col1, _col2, _col3
- Group By Operator [GBY_531] (rows=736356923 width=131)
+ Group By Operator [GBY_591] (rows=736356923 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],keys:_col0, _col1, _col2, _col3, _col4, _col5
- Select Operator [SEL_529] (rows=115177968 width=131)
+ Select Operator [SEL_589] (rows=115177968 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Merge Join Operator [MERGEJOIN_528] (rows=115177968 width=220)
- Conds:RS_66._col1, _col2=RS_627._col0, _col1(Left Outer),Output:["_col3","_col4","_col7","_col8","_col9","_col10","_col13","_col14"]
+ Merge Join Operator [MERGEJOIN_588] (rows=115177968 width=220)
+ Conds:RS_66._col1, _col2=RS_687._col0, _col1(Left Outer),Output:["_col3","_col4","_col7","_col8","_col9","_col10","_col13","_col14"]
<-Map 42 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_627]
+ SHUFFLE [RS_687]
PartitionCols:_col0, _col1
- Please refer to the previous Select Operator [SEL_626]
+ Please refer to the previous Select Operator [SEL_686]
<-Reducer 18 [SIMPLE_EDGE]
SHUFFLE [RS_66]
PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_502] (rows=48990732 width=139)
- Conds:RS_63._col1=RS_593._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col7","_col8","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_562] (rows=48990732 width=139)
+ Conds:RS_63._col1=RS_653._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col7","_col8","_col9","_col10"]
<-Map 37 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_593]
+ SHUFFLE [RS_653]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_590]
+ Please refer to the previous Select Operator [SEL_650]
<-Reducer 17 [SIMPLE_EDGE]
SHUFFLE [RS_63]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_501] (rows=51404771 width=123)
- Conds:RS_625._col0=RS_564._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_561] (rows=51404771 width=123)
+ Conds:RS_685._col0=RS_624._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
<-Map 11 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_564]
+ PARTITION_ONLY_SHUFFLE [RS_624]
PartitionCols:_col0
- Select Operator [SEL_556] (rows=652 width=4)
+ Select Operator [SEL_616] (rows=652 width=4)
Output:["_col0"]
- Filter Operator [FIL_552] (rows=652 width=8)
+ Filter Operator [FIL_612] (rows=652 width=8)
predicate:(d_year = 2001)
Please refer to the previous TableScan [TS_3]
<-Map 41 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_625]
+ SHUFFLE [RS_685]
PartitionCols:_col0
- Select Operator [SEL_624] (rows=143966864 width=127)
+ Select Operator [SEL_684] (rows=143966864 width=127)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_623] (rows=143966864 width=127)
+ Filter Operator [FIL_683] (rows=143966864 width=127)
predicate:(ws_sold_date_sk is not null and ws_sold_date_sk BETWEEN DynamicValue(RS_61_date_dim_d_date_sk_min) AND DynamicValue(RS_61_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_61_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_49] (rows=144002668 width=127)
default@web_sales,web_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ws_sold_date_sk","ws_item_sk","ws_order_number","ws_quantity","ws_ext_sales_price"]
<-Reducer 20 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_622]
- Group By Operator [GBY_621] (rows=1 width=12)
+ BROADCAST [RS_682]
+ Group By Operator [GBY_681] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 11 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_580]
- Group By Operator [GBY_574] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_640]
+ Group By Operator [GBY_634] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_565] (rows=652 width=4)
+ Select Operator [SEL_625] (rows=652 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_556]
+ Please refer to the previous Select Operator [SEL_616]
<-Reducer 6 [CONTAINS] vectorized
- Reduce Output Operator [RS_602]
+ Reduce Output Operator [RS_662]
PartitionCols:_col0, _col1, _col2, _col3
- Group By Operator [GBY_601] (rows=736356923 width=131)
+ Group By Operator [GBY_661] (rows=736356923 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],keys:_col0, _col1, _col2, _col3, _col4, _col5
- Group By Operator [GBY_600] (rows=621178955 width=131)
+ Group By Operator [GBY_660] (rows=621178955 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5
<-Union 5 [SIMPLE_EDGE]
<-Reducer 15 [CONTAINS]
- Reduce Output Operator [RS_527]
+ Reduce Output Operator [RS_587]
PartitionCols:_col0, _col1, _col2, _col3, _col4, _col5
- Group By Operator [GBY_526] (rows=621178955 width=131)
+ Group By Operator [GBY_586] (rows=621178955 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],keys:_col0, _col1, _col2, _col3, _col4, _col5
- Select Operator [SEL_524] (rows=450703984 width=131)
+ Select Operator [SEL_584] (rows=450703984 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Merge Join Operator [MERGEJOIN_523] (rows=450703984 width=204)
- Conds:RS_38._col1, _col2=RS_619._col0, _col1(Left Outer),Output:["_col3","_col4","_col7","_col8","_col9","_col10","_col13","_col14"]
+ Merge Join Operator [MERGEJOIN_583] (rows=450703984 width=204)
+ Conds:RS_38._col1, _col2=RS_679._col0, _col1(Left Outer),Output:["_col3","_col4","_col7","_col8","_col9","_col10","_col13","_col14"]
<-Map 40 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_619]
+ SHUFFLE [RS_679]
PartitionCols:_col0, _col1
- Please refer to the previous Select Operator [SEL_618]
+ Please refer to the previous Select Operator [SEL_678]
<-Reducer 14 [SIMPLE_EDGE]
SHUFFLE [RS_38]
PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_499] (rows=187186493 width=124)
- Conds:RS_35._col1=RS_592._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col7","_col8","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_559] (rows=187186493 width=124)
+ Conds:RS_35._col1=RS_652._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col7","_col8","_col9","_col10"]
<-Map 37 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_592]
+ SHUFFLE [RS_652]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_590]
+ Please refer to the previous Select Operator [SEL_650]
<-Reducer 13 [SIMPLE_EDGE]
SHUFFLE [RS_35]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_498] (rows=196410188 width=109)
- Conds:RS_617._col0=RS_562._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_558] (rows=196410188 width=109)
+ Conds:RS_677._col0=RS_622._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
<-Map 11 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_562]
+ PARTITION_ONLY_SHUFFLE [RS_622]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_556]
+ Please refer to the previous Select Operator [SEL_616]
<-Map 39 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_617]
+ SHUFFLE [RS_677]
PartitionCols:_col0
- Select Operator [SEL_616] (rows=550076554 width=122)
+ Select Operator [SEL_676] (rows=550076554 width=122)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_615] (rows=550076554 width=122)
+ Filter Operator [FIL_675] (rows=550076554 width=122)
predicate:(ss_sold_date_sk is not null and ss_sold_date_sk BETWEEN DynamicValue(RS_33_date_dim_d_date_sk_min) AND DynamicValue(RS_33_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_33_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_21] (rows=575995635 width=122)
default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_item_sk","ss_ticket_number","ss_quantity","ss_ext_sales_price"]
<-Reducer 16 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_614]
- Group By Operator [GBY_613] (rows=1 width=12)
+ BROADCAST [RS_674]
+ Group By Operator [GBY_673] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 11 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_579]
- Group By Operator [GBY_573] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_639]
+ Group By Operator [GBY_633] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_563] (rows=652 width=4)
+ Select Operator [SEL_623] (rows=652 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_556]
+ Please refer to the previous Select Operator [SEL_616]
<-Reducer 4 [CONTAINS]
- Reduce Output Operator [RS_518]
+ Reduce Output Operator [RS_578]
PartitionCols:_col0, _col1, _col2, _col3, _col4, _col5
- Group By Operator [GBY_517] (rows=621178955 width=131)
+ Group By Operator [GBY_577] (rows=621178955 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],keys:_col0, _col1, _col2, _col3, _col4, _col5
- Select Operator [SEL_515] (rows=170474971 width=131)
+ Select Operator [SEL_575] (rows=170474971 width=131)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Merge Join Operator [MERGEJOIN_514] (rows=170474971 width=234)
- Conds:RS_17._col1, _col2=RS_598._col0, _col1(Left Outer),Output:["_col3","_col4","_col7","_col8","_col9","_col10","_col13","_col14"]
+ Merge Join Operator [MERGEJOIN_574] (rows=170474971 width=234)
+ Conds:RS_17._col1, _col2=RS_658._col0, _col1(Left Outer),Output:["_col3","_col4","_col7","_col8","_col9","_col10","_col13","_col14"]
<-Map 38 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_598]
+ SHUFFLE [RS_658]
PartitionCols:_col0, _col1
- Please refer to the previous Select Operator [SEL_597]
+ Please refer to the previous Select Operator [SEL_657]
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_17]
PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_496] (rows=96821196 width=138)
- Conds:RS_14._col1=RS_591._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col7","_col8","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_556] (rows=96821196 width=138)
+ Conds:RS_14._col1=RS_651._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col7","_col8","_col9","_col10"]
<-Map 37 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_591]
+ SHUFFLE [RS_651]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_590]
+ Please refer to the previous Select Operator [SEL_650]
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_14]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_495] (rows=101592102 width=122)
- Conds:RS_588._col0=RS_560._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_555] (rows=101592102 width=122)
+ Conds:RS_648._col0=RS_620._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
<-Map 11 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_560]
+ PARTITION_ONLY_SHUFFLE [RS_620]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_556]
+ Please refer to the previous Select Operator [SEL_616]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_588]
+ SHUFFLE [RS_648]
PartitionCols:_col0
- Select Operator [SEL_587] (rows=286549727 width=127)
+ Select Operator [SEL_647] (rows=286549727 width=127)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_586] (rows=286549727 width=127)
+ Filter Operator [FIL_646] (rows=286549727 width=127)
predicate:(cs_sold_date_sk is not null and cs_sold_date_sk BETWEEN DynamicValue(RS_12_date_dim_d_date_sk_min) AND DynamicValue(RS_12_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_12_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_0] (rows=287989836 width=127)
default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["cs_sold_date_sk","cs_item_sk","cs_order_number","cs_quantity","cs_ext_sales_price"]
<-Reducer 12 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_585]
- Group By Operator [GBY_584] (rows=1 width=12)
+ BROADCAST [RS_645]
+ Group By Operator [GBY_644] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 11 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_578]
- Group By Operator [GBY_572] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_638]
+ Group By Operator [GBY_632] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_561] (rows=652 width=4)
+ Select Operator [SEL_621] (rows=652 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_556]
+ Please refer to the previous Select Operator [SEL_616]
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query78.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query78.q.out
index 0114949..9d667aa 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query78.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query78.q.out
@@ -131,33 +131,34 @@ POSTHOOK: Output: hdfs://### HDFS PATH ###
Plan optimized by CBO.
Vertex dependency in root stage
-Map 13 <- Reducer 11 (BROADCAST_EDGE)
-Map 16 <- Reducer 11 (BROADCAST_EDGE)
-Map 19 <- Reducer 12 (BROADCAST_EDGE)
+Map 14 <- Reducer 12 (BROADCAST_EDGE)
+Map 17 <- Reducer 11 (BROADCAST_EDGE)
+Map 20 <- Reducer 13 (BROADCAST_EDGE)
Reducer 10 <- Reducer 9 (SIMPLE_EDGE)
Reducer 11 <- Reducer 10 (CUSTOM_SIMPLE_EDGE)
-Reducer 12 <- Map 1 (CUSTOM_SIMPLE_EDGE)
-Reducer 14 <- Map 13 (SIMPLE_EDGE), Map 15 (SIMPLE_EDGE)
-Reducer 17 <- Map 16 (SIMPLE_EDGE), Map 18 (SIMPLE_EDGE)
-Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 14 (SIMPLE_EDGE)
-Reducer 20 <- Map 19 (SIMPLE_EDGE), Map 21 (SIMPLE_EDGE)
+Reducer 12 <- Reducer 10 (CUSTOM_SIMPLE_EDGE)
+Reducer 13 <- Map 1 (CUSTOM_SIMPLE_EDGE)
+Reducer 15 <- Map 14 (SIMPLE_EDGE), Map 16 (SIMPLE_EDGE)
+Reducer 18 <- Map 17 (SIMPLE_EDGE), Map 19 (SIMPLE_EDGE)
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 15 (SIMPLE_EDGE)
+Reducer 21 <- Map 20 (SIMPLE_EDGE), Map 22 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
Reducer 4 <- Reducer 3 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
Reducer 5 <- Reducer 10 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 6 <- Reducer 5 (SIMPLE_EDGE)
-Reducer 7 <- Map 1 (SIMPLE_EDGE), Reducer 17 (SIMPLE_EDGE)
+Reducer 7 <- Map 1 (SIMPLE_EDGE), Reducer 18 (SIMPLE_EDGE)
Reducer 8 <- Reducer 7 (SIMPLE_EDGE)
-Reducer 9 <- Map 1 (SIMPLE_EDGE), Reducer 20 (SIMPLE_EDGE)
+Reducer 9 <- Map 1 (SIMPLE_EDGE), Reducer 21 (SIMPLE_EDGE)
Stage-0
Fetch Operator
limit:100
Stage-1
Reducer 6 vectorized
- File Output Operator [FS_272]
- Limit [LIM_271] (rows=100 width=484)
+ File Output Operator [FS_301]
+ Limit [LIM_300] (rows=100 width=484)
Number of rows:100
- Select Operator [SEL_270] (rows=75385523694 width=483)
+ Select Operator [SEL_299] (rows=75385523694 width=483)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_76]
@@ -165,173 +166,181 @@ Stage-0
Output:["_col0","_col1","_col6","_col7","_col8","_col9","_col10","_col11","_col12"]
Top N Key Operator [TNK_117] (rows=75385523694 width=703)
keys:_col0, _col1, _col2, _col3, _col4, (CASE WHEN (_col11 is not null) THEN (_col11) ELSE (0L) END + CASE WHEN (_col6 is not null) THEN (_col6) ELSE (0L) END), (CASE WHEN (_col12 is not null) THEN (_col12) ELSE (0) END + CASE WHEN (_col7 is not null) THEN (_col7) ELSE (0) END), (CASE WHEN (_col13 is not null) THEN (_col13) ELSE (0) END + CASE WHEN (_col8 is not null) THEN (_col8) ELSE (0) END), round((UDFToDouble(_col2) / UDFToDouble(CASE WHEN ((_col11 is not null and [...]
- Merge Join Operator [MERGEJOIN_224] (rows=75385523694 width=703)
- Conds:RS_72._col0, _col1=RS_243._col0, _col1(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col6","_col7","_col8","_col11","_col12","_col13"]
+ Merge Join Operator [MERGEJOIN_249] (rows=75385523694 width=703)
+ Conds:RS_72._col0, _col1=RS_268._col0, _col1(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col6","_col7","_col8","_col11","_col12","_col13"]
<-Reducer 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_243]
+ SHUFFLE [RS_268]
PartitionCols:_col0, _col1
- Filter Operator [FIL_242] (rows=1008632 width=239)
+ Filter Operator [FIL_267] (rows=1008632 width=239)
predicate:(_col2 > 0L)
- Group By Operator [GBY_241] (rows=1008632 width=239)
+ Group By Operator [GBY_266] (rows=1008632 width=239)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0, KEY._col1
<-Reducer 9 [SIMPLE_EDGE]
SHUFFLE [RS_65]
PartitionCols:_col0, _col1
Group By Operator [GBY_64] (rows=1008632 width=239)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col4)","sum(_col5)","sum(_col6)"],keys:_col2, _col3
- Merge Join Operator [MERGEJOIN_222] (rows=1008632 width=227)
- Conds:RS_229._col0=RS_61._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col6"]
+ Merge Join Operator [MERGEJOIN_247] (rows=1008632 width=227)
+ Conds:RS_254._col0=RS_61._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col6"]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_229]
+ SHUFFLE [RS_254]
PartitionCols:_col0
- Select Operator [SEL_226] (rows=652 width=4)
+ Select Operator [SEL_251] (rows=652 width=4)
Output:["_col0"]
- Filter Operator [FIL_225] (rows=652 width=8)
+ Filter Operator [FIL_250] (rows=652 width=8)
predicate:(d_year = 2000)
TableScan [TS_0] (rows=73049 width=8)
default@date_dim,date_dim,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_year"]
- <-Reducer 20 [SIMPLE_EDGE]
+ <-Reducer 21 [SIMPLE_EDGE]
SHUFFLE [RS_61]
PartitionCols:_col0
Select Operator [SEL_59] (rows=2824825 width=236)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Merge Join Operator [MERGEJOIN_221] (rows=2824825 width=236)
- Conds:RS_237._col1, _col3=RS_240._col0, _col1(Anti),Output:["_col0","_col1","_col2","_col4","_col5","_col6"]
- <-Map 19 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_237]
+ Merge Join Operator [MERGEJOIN_246] (rows=2824825 width=236)
+ Conds:RS_262._col1, _col3=RS_265._col0, _col1(Anti),Output:["_col0","_col1","_col2","_col4","_col5","_col6"]
+ <-Map 20 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_262]
PartitionCols:_col1, _col3
- Select Operator [SEL_236] (rows=143930993 width=243)
+ Select Operator [SEL_261] (rows=143930993 width=243)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- Filter Operator [FIL_235] (rows=143930993 width=243)
+ Filter Operator [FIL_260] (rows=143930993 width=243)
predicate:(ws_bill_customer_sk is not null and ws_sold_date_sk is not null and ws_sold_date_sk BETWEEN DynamicValue(RS_60_date_dim_d_date_sk_min) AND DynamicValue(RS_60_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_60_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_49] (rows=144002668 width=243)
default@web_sales,web_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ws_sold_date_sk","ws_item_sk","ws_bill_customer_sk","ws_order_number","ws_quantity","ws_wholesale_cost","ws_sales_price"]
- <-Reducer 12 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_234]
- Group By Operator [GBY_233] (rows=1 width=12)
+ <-Reducer 13 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_259]
+ Group By Operator [GBY_258] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 1 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_232]
- Group By Operator [GBY_231] (rows=1 width=12)
+ SHUFFLE [RS_257]
+ Group By Operator [GBY_256] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_230] (rows=652 width=4)
+ Select Operator [SEL_255] (rows=652 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_226]
- <-Map 21 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_240]
+ Please refer to the previous Select Operator [SEL_251]
+ <-Map 22 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_265]
PartitionCols:_col0, _col1
- Group By Operator [GBY_239] (rows=14398467 width=8)
+ Group By Operator [GBY_264] (rows=14398467 width=8)
Output:["_col0","_col1"],keys:_col0, _col1
- Select Operator [SEL_238] (rows=14398467 width=8)
+ Select Operator [SEL_263] (rows=14398467 width=8)
Output:["_col0","_col1"]
TableScan [TS_52] (rows=14398467 width=8)
default@web_returns,web_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["wr_item_sk","wr_order_number"]
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_72]
PartitionCols:_col0, _col1
- Merge Join Operator [MERGEJOIN_223] (rows=13664108245 width=471)
- Conds:RS_258._col1=RS_269._col0(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col6","_col7","_col8"]
+ Merge Join Operator [MERGEJOIN_248] (rows=13664108245 width=471)
+ Conds:RS_285._col1=RS_298._col0(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col6","_col7","_col8"]
<-Reducer 3 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_258]
+ SHUFFLE [RS_285]
PartitionCols:_col1
- Select Operator [SEL_257] (rows=187573258 width=239)
+ Select Operator [SEL_284] (rows=187573258 width=239)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_256] (rows=187573258 width=239)
+ Group By Operator [GBY_283] (rows=187573258 width=239)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0, KEY._col1
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_19]
PartitionCols:_col0, _col1
Group By Operator [GBY_18] (rows=187573258 width=239)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col4)","sum(_col5)","sum(_col6)"],keys:_col3, _col2
- Merge Join Operator [MERGEJOIN_218] (rows=187573258 width=206)
- Conds:RS_227._col0=RS_15._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col6"]
+ Merge Join Operator [MERGEJOIN_243] (rows=187573258 width=206)
+ Conds:RS_252._col0=RS_15._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col6"]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_227]
+ SHUFFLE [RS_252]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_226]
- <-Reducer 14 [SIMPLE_EDGE]
+ Please refer to the previous Select Operator [SEL_251]
+ <-Reducer 15 [SIMPLE_EDGE]
SHUFFLE [RS_15]
PartitionCols:_col0
Select Operator [SEL_13] (rows=525327388 width=229)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Merge Join Operator [MERGEJOIN_217] (rows=525327388 width=229)
- Conds:RS_252._col1, _col3=RS_255._col0, _col1(Anti),Output:["_col0","_col1","_col2","_col4","_col5","_col6"]
- <-Map 13 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_252]
+ Merge Join Operator [MERGEJOIN_242] (rows=525327388 width=229)
+ Conds:RS_279._col1, _col3=RS_282._col0, _col1(Anti),Output:["_col0","_col1","_col2","_col4","_col5","_col6"]
+ <-Map 14 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_279]
PartitionCols:_col1, _col3
- Select Operator [SEL_251] (rows=525327388 width=233)
+ Select Operator [SEL_278] (rows=525327388 width=233)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- Filter Operator [FIL_250] (rows=525327388 width=233)
- predicate:(ss_sold_date_sk is not null and ss_customer_sk is not null and ss_customer_sk BETWEEN DynamicValue(RS_73_web_sales_ws_bill_customer_sk_min) AND DynamicValue(RS_73_web_sales_ws_bill_customer_sk_max) and in_bloom_filter(ss_customer_sk, DynamicValue(RS_73_web_sales_ws_bill_customer_sk_bloom_filter)))
+ Filter Operator [FIL_277] (rows=525327388 width=233)
+ predicate:(ss_sold_date_sk is not null and ss_customer_sk is not null and ss_item_sk BETWEEN DynamicValue(RS[221]_col0) AND DynamicValue(RS[221]_col1) and ss_customer_sk BETWEEN DynamicValue(RS[221]_col2) AND DynamicValue(RS[221]_col3) and in_bloom_filter(hash(ss_item_sk,ss_customer_sk), DynamicValue(RS[221]_col4)))
TableScan [TS_3] (rows=575995635 width=233)
default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_item_sk","ss_customer_sk","ss_ticket_number","ss_quantity","ss_wholesale_cost","ss_sales_price"]
- <-Reducer 11 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_248]
- Group By Operator [GBY_247] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Reducer 12 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_276]
+ Group By Operator [GBY_275] (rows=1 width=160)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","min(VALUE._col2)","max(VALUE._col3)","bloom_filter(VALUE._col4, expectedEntries=1000000)"]
<-Reducer 10 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_246]
- Group By Operator [GBY_245] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_244] (rows=1008632 width=7)
- Output:["_col0"]
- Please refer to the previous Filter Operator [FIL_242]
- <-Map 15 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_255]
+ SHUFFLE [RS_274]
+ Group By Operator [GBY_272] (rows=1 width=160)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["min(_col0)","max(_col0)","min(_col1)","max(_col1)","bloom_filter(_col3, expectedEntries=1000000)"]
+ Select Operator [SEL_270] (rows=1008632 width=11)
+ Output:["_col0","_col1","_col3"]
+ Please refer to the previous Filter Operator [FIL_267]
+ <-Map 16 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_282]
PartitionCols:_col0, _col1
- Group By Operator [GBY_254] (rows=57591150 width=8)
+ Group By Operator [GBY_281] (rows=57591150 width=8)
Output:["_col0","_col1"],keys:_col0, _col1
- Select Operator [SEL_253] (rows=57591150 width=8)
+ Select Operator [SEL_280] (rows=57591150 width=8)
Output:["_col0","_col1"]
TableScan [TS_6] (rows=57591150 width=8)
default@store_returns,store_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["sr_item_sk","sr_ticket_number"]
<-Reducer 8 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_269]
+ SHUFFLE [RS_298]
PartitionCols:_col0
- Select Operator [SEL_268] (rows=36821353 width=235)
+ Select Operator [SEL_297] (rows=36821353 width=235)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_267] (rows=36821353 width=239)
+ Filter Operator [FIL_296] (rows=36821353 width=239)
predicate:(_col2 > 0L)
- Select Operator [SEL_266] (rows=36821353 width=239)
+ Select Operator [SEL_295] (rows=36821353 width=239)
Output:["_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_265] (rows=36821353 width=239)
+ Group By Operator [GBY_294] (rows=36821353 width=239)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0, KEY._col1
<-Reducer 7 [SIMPLE_EDGE]
SHUFFLE [RS_41]
PartitionCols:_col0, _col1
Group By Operator [GBY_40] (rows=36821353 width=239)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col4)","sum(_col5)","sum(_col6)"],keys:_col2, _col3
- Merge Join Operator [MERGEJOIN_220] (rows=36821353 width=227)
- Conds:RS_228._col0=RS_37._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col6"]
+ Merge Join Operator [MERGEJOIN_245] (rows=36821353 width=227)
+ Conds:RS_253._col0=RS_37._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col6"]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_228]
+ SHUFFLE [RS_253]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_226]
- <-Reducer 17 [SIMPLE_EDGE]
+ Please refer to the previous Select Operator [SEL_251]
+ <-Reducer 18 [SIMPLE_EDGE]
SHUFFLE [RS_37]
PartitionCols:_col0
Select Operator [SEL_35] (rows=103857961 width=236)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Merge Join Operator [MERGEJOIN_219] (rows=103857961 width=236)
- Conds:RS_261._col2, _col3=RS_264._col0, _col1(Anti),Output:["_col0","_col1","_col2","_col4","_col5","_col6"]
- <-Map 16 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_261]
+ Merge Join Operator [MERGEJOIN_244] (rows=103857961 width=236)
+ Conds:RS_290._col2, _col3=RS_293._col0, _col1(Anti),Output:["_col0","_col1","_col2","_col4","_col5","_col6"]
+ <-Map 17 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_290]
PartitionCols:_col2, _col3
- Select Operator [SEL_260] (rows=285117831 width=242)
+ Select Operator [SEL_289] (rows=285117831 width=242)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- Filter Operator [FIL_259] (rows=285117831 width=242)
+ Filter Operator [FIL_288] (rows=285117831 width=242)
predicate:(cs_sold_date_sk is not null and cs_bill_customer_sk is not null and cs_bill_customer_sk BETWEEN DynamicValue(RS_73_web_sales_ws_bill_customer_sk_min) AND DynamicValue(RS_73_web_sales_ws_bill_customer_sk_max) and in_bloom_filter(cs_bill_customer_sk, DynamicValue(RS_73_web_sales_ws_bill_customer_sk_bloom_filter)))
TableScan [TS_25] (rows=287989836 width=242)
default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["cs_sold_date_sk","cs_bill_customer_sk","cs_item_sk","cs_order_number","cs_quantity","cs_wholesale_cost","cs_sales_price"]
<-Reducer 11 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_249]
- Please refer to the previous Group By Operator [GBY_247]
- <-Map 18 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_264]
+ BROADCAST [RS_287]
+ Group By Operator [GBY_286] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Reducer 10 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_273]
+ Group By Operator [GBY_271] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_269] (rows=1008632 width=7)
+ Output:["_col0"]
+ Please refer to the previous Filter Operator [FIL_267]
+ <-Map 19 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_293]
PartitionCols:_col0, _col1
- Group By Operator [GBY_263] (rows=28798881 width=8)
+ Group By Operator [GBY_292] (rows=28798881 width=8)
Output:["_col0","_col1"],keys:_col0, _col1
- Select Operator [SEL_262] (rows=28798881 width=8)
+ Select Operator [SEL_291] (rows=28798881 width=8)
Output:["_col0","_col1"]
TableScan [TS_28] (rows=28798881 width=8)
default@catalog_returns,catalog_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["cr_item_sk","cr_order_number"]
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query80.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query80.q.out
index c90b253..1b409d0 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query80.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query80.q.out
@@ -249,28 +249,28 @@ Stage-0
limit:100
Stage-1
Reducer 10 vectorized
- File Output Operator [FS_435]
- Limit [LIM_434] (rows=100 width=619)
+ File Output Operator [FS_450]
+ Limit [LIM_449] (rows=100 width=619)
Number of rows:100
- Select Operator [SEL_433] (rows=59581 width=619)
+ Select Operator [SEL_448] (rows=59581 width=619)
Output:["_col0","_col1","_col2","_col3","_col4"]
<-Reducer 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_432]
- Select Operator [SEL_431] (rows=59581 width=619)
+ SHUFFLE [RS_447]
+ Select Operator [SEL_446] (rows=59581 width=619)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_430] (rows=59581 width=627)
+ Group By Operator [GBY_445] (rows=59581 width=627)
Output:["_col0","_col1","_col3","_col4","_col5"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Union 8 [SIMPLE_EDGE]
<-Reducer 18 [CONTAINS] vectorized
- Reduce Output Operator [RS_449]
+ Reduce Output Operator [RS_464]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_448] (rows=59581 width=627)
+ Group By Operator [GBY_463] (rows=59581 width=627)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0, _col1, 0L
- Top N Key Operator [TNK_447] (rows=39721 width=618)
+ Top N Key Operator [TNK_462] (rows=39721 width=618)
keys:_col0, _col1,top n:100
- Select Operator [SEL_446] (rows=38846 width=619)
+ Select Operator [SEL_461] (rows=38846 width=619)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_445] (rows=38846 width=436)
+ Group By Operator [GBY_460] (rows=38846 width=436)
Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0
<-Reducer 17 [SIMPLE_EDGE]
SHUFFLE [RS_71]
@@ -279,99 +279,99 @@ Stage-0
Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)"],keys:_col0
Select Operator [SEL_68] (rows=8592843 width=305)
Output:["_col0","_col1","_col2","_col3"]
- Merge Join Operator [MERGEJOIN_368] (rows=8592843 width=305)
- Conds:RS_65._col1=RS_444._col0(Inner),Output:["_col5","_col6","_col9","_col10","_col15"]
+ Merge Join Operator [MERGEJOIN_383] (rows=8592843 width=305)
+ Conds:RS_65._col1=RS_459._col0(Inner),Output:["_col5","_col6","_col9","_col10","_col15"]
<-Map 32 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_444]
+ SHUFFLE [RS_459]
PartitionCols:_col0
- Select Operator [SEL_443] (rows=46000 width=104)
+ Select Operator [SEL_458] (rows=46000 width=104)
Output:["_col0","_col1"]
TableScan [TS_51] (rows=46000 width=104)
default@catalog_page,catalog_page,Tbl:COMPLETE,Col:COMPLETE,Output:["cp_catalog_page_sk","cp_catalog_page_id"]
<-Reducer 16 [SIMPLE_EDGE]
SHUFFLE [RS_65]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_367] (rows=8592843 width=208)
- Conds:RS_62._col3=RS_421._col0(Inner),Output:["_col1","_col5","_col6","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_382] (rows=8592843 width=208)
+ Conds:RS_62._col3=RS_436._col0(Inner),Output:["_col1","_col5","_col6","_col9","_col10"]
<-Map 27 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_421]
+ SHUFFLE [RS_436]
PartitionCols:_col0
- Select Operator [SEL_419] (rows=1150 width=4)
+ Select Operator [SEL_434] (rows=1150 width=4)
Output:["_col0"]
- Filter Operator [FIL_418] (rows=1150 width=89)
+ Filter Operator [FIL_433] (rows=1150 width=89)
predicate:(p_channel_tv = 'N')
TableScan [TS_11] (rows=2300 width=89)
default@promotion,promotion,Tbl:COMPLETE,Col:COMPLETE,Output:["p_promo_sk","p_channel_tv"]
<-Reducer 15 [SIMPLE_EDGE]
SHUFFLE [RS_62]
PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_366] (rows=17185686 width=222)
- Conds:RS_59._col2=RS_416._col0(Inner),Output:["_col1","_col3","_col5","_col6","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_381] (rows=17185686 width=222)
+ Conds:RS_59._col2=RS_431._col0(Inner),Output:["_col1","_col3","_col5","_col6","_col9","_col10"]
<-Map 26 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_416]
+ SHUFFLE [RS_431]
PartitionCols:_col0
- Select Operator [SEL_414] (rows=154000 width=4)
+ Select Operator [SEL_429] (rows=154000 width=4)
Output:["_col0"]
- Filter Operator [FIL_413] (rows=154000 width=115)
+ Filter Operator [FIL_428] (rows=154000 width=115)
predicate:(i_current_price > 50)
TableScan [TS_8] (rows=462000 width=115)
default@item,item,Tbl:COMPLETE,Col:COMPLETE,Output:["i_item_sk","i_current_price"]
<-Reducer 14 [SIMPLE_EDGE]
SHUFFLE [RS_59]
PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_365] (rows=51557056 width=232)
- Conds:RS_56._col0=RS_396._col0(Inner),Output:["_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_380] (rows=51557056 width=232)
+ Conds:RS_56._col0=RS_411._col0(Inner),Output:["_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
<-Map 12 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_396]
+ SHUFFLE [RS_411]
PartitionCols:_col0
- Select Operator [SEL_393] (rows=8116 width=4)
+ Select Operator [SEL_408] (rows=8116 width=4)
Output:["_col0"]
- Filter Operator [FIL_392] (rows=8116 width=98)
+ Filter Operator [FIL_407] (rows=8116 width=98)
predicate:CAST( d_date AS TIMESTAMP) BETWEEN TIMESTAMP'1998-08-04 00:00:00' AND TIMESTAMP'1998-09-03 00:00:00'
TableScan [TS_5] (rows=73049 width=98)
default@date_dim,date_dim,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_date"]
<-Reducer 30 [SIMPLE_EDGE]
SHUFFLE [RS_56]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_364] (rows=464045263 width=326)
- Conds:RS_440._col2, _col4=RS_442._col0, _col1(Left Outer),Output:["_col0","_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_379] (rows=464045263 width=326)
+ Conds:RS_455._col2, _col4=RS_457._col0, _col1(Left Outer),Output:["_col0","_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
<-Map 29 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_440]
+ SHUFFLE [RS_455]
PartitionCols:_col2, _col4
- Select Operator [SEL_439] (rows=283691906 width=243)
+ Select Operator [SEL_454] (rows=283691906 width=243)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- Filter Operator [FIL_438] (rows=283691906 width=243)
+ Filter Operator [FIL_453] (rows=283691906 width=243)
predicate:(cs_promo_sk is not null and cs_sold_date_sk is not null and cs_catalog_page_sk is not null and cs_sold_date_sk BETWEEN DynamicValue(RS_57_date_dim_d_date_sk_min) AND DynamicValue(RS_57_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_57_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_37] (rows=287989836 width=243)
default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["cs_sold_date_sk","cs_catalog_page_sk","cs_item_sk","cs_promo_sk","cs_order_number","cs_ext_sales_price","cs_net_profit"]
<-Reducer 19 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_437]
- Group By Operator [GBY_436] (rows=1 width=12)
+ BROADCAST [RS_452]
+ Group By Operator [GBY_451] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_404]
- Group By Operator [GBY_401] (rows=1 width=12)
+ SHUFFLE [RS_419]
+ Group By Operator [GBY_416] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_397] (rows=8116 width=4)
+ Select Operator [SEL_412] (rows=8116 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_393]
+ Please refer to the previous Select Operator [SEL_408]
<-Map 31 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_442]
+ SHUFFLE [RS_457]
PartitionCols:_col0, _col1
- Select Operator [SEL_441] (rows=28798881 width=227)
+ Select Operator [SEL_456] (rows=28798881 width=227)
Output:["_col0","_col1","_col2","_col3"]
TableScan [TS_40] (rows=28798881 width=227)
default@catalog_returns,catalog_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["cr_item_sk","cr_order_number","cr_return_amount","cr_net_loss"]
<-Reducer 24 [CONTAINS] vectorized
- Reduce Output Operator [RS_463]
+ Reduce Output Operator [RS_478]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_462] (rows=59581 width=627)
+ Group By Operator [GBY_477] (rows=59581 width=627)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0, _col1, 0L
- Top N Key Operator [TNK_461] (rows=39721 width=618)
+ Top N Key Operator [TNK_476] (rows=39721 width=618)
keys:_col0, _col1,top n:100
- Select Operator [SEL_460] (rows=53 width=615)
+ Select Operator [SEL_475] (rows=53 width=615)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_459] (rows=53 width=436)
+ Group By Operator [GBY_474] (rows=53 width=436)
Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0
<-Reducer 23 [SIMPLE_EDGE]
SHUFFLE [RS_109]
@@ -380,84 +380,84 @@ Stage-0
Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)"],keys:_col0
Select Operator [SEL_106] (rows=4714659 width=323)
Output:["_col0","_col1","_col2","_col3"]
- Merge Join Operator [MERGEJOIN_373] (rows=4714659 width=323)
- Conds:RS_103._col2=RS_458._col0(Inner),Output:["_col5","_col6","_col9","_col10","_col15"]
+ Merge Join Operator [MERGEJOIN_388] (rows=4714659 width=323)
+ Conds:RS_103._col2=RS_473._col0(Inner),Output:["_col5","_col6","_col9","_col10","_col15"]
<-Map 36 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_458]
+ SHUFFLE [RS_473]
PartitionCols:_col0
- Select Operator [SEL_457] (rows=84 width=104)
+ Select Operator [SEL_472] (rows=84 width=104)
Output:["_col0","_col1"]
TableScan [TS_89] (rows=84 width=104)
default@web_site,web_site,Tbl:COMPLETE,Col:COMPLETE,Output:["web_site_sk","web_site_id"]
<-Reducer 22 [SIMPLE_EDGE]
SHUFFLE [RS_103]
PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_372] (rows=4714659 width=227)
- Conds:RS_100._col3=RS_422._col0(Inner),Output:["_col2","_col5","_col6","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_387] (rows=4714659 width=227)
+ Conds:RS_100._col3=RS_437._col0(Inner),Output:["_col2","_col5","_col6","_col9","_col10"]
<-Map 27 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_422]
+ SHUFFLE [RS_437]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_419]
+ Please refer to the previous Select Operator [SEL_434]
<-Reducer 21 [SIMPLE_EDGE]
SHUFFLE [RS_100]
PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_371] (rows=9429318 width=231)
- Conds:RS_97._col1=RS_417._col0(Inner),Output:["_col2","_col3","_col5","_col6","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_386] (rows=9429318 width=231)
+ Conds:RS_97._col1=RS_432._col0(Inner),Output:["_col2","_col3","_col5","_col6","_col9","_col10"]
<-Map 26 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_417]
+ SHUFFLE [RS_432]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_414]
+ Please refer to the previous Select Operator [SEL_429]
<-Reducer 20 [SIMPLE_EDGE]
SHUFFLE [RS_97]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_370] (rows=28287952 width=235)
- Conds:RS_94._col0=RS_398._col0(Inner),Output:["_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_385] (rows=28287952 width=235)
+ Conds:RS_94._col0=RS_413._col0(Inner),Output:["_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
<-Map 12 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_398]
+ SHUFFLE [RS_413]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_393]
+ Please refer to the previous Select Operator [SEL_408]
<-Reducer 34 [SIMPLE_EDGE]
SHUFFLE [RS_94]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_369] (rows=254608997 width=363)
- Conds:RS_454._col1, _col4=RS_456._col0, _col1(Left Outer),Output:["_col0","_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_384] (rows=254608997 width=363)
+ Conds:RS_469._col1, _col4=RS_471._col0, _col1(Left Outer),Output:["_col0","_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
<-Map 33 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_454]
+ SHUFFLE [RS_469]
PartitionCols:_col1, _col4
- Select Operator [SEL_453] (rows=143894769 width=243)
+ Select Operator [SEL_468] (rows=143894769 width=243)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- Filter Operator [FIL_452] (rows=143894769 width=243)
+ Filter Operator [FIL_467] (rows=143894769 width=243)
predicate:(ws_promo_sk is not null and ws_web_site_sk is not null and ws_sold_date_sk is not null and ws_sold_date_sk BETWEEN DynamicValue(RS_95_date_dim_d_date_sk_min) AND DynamicValue(RS_95_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_95_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_75] (rows=144002668 width=243)
default@web_sales,web_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ws_sold_date_sk","ws_item_sk","ws_web_site_sk","ws_promo_sk","ws_order_number","ws_ext_sales_price","ws_net_profit"]
<-Reducer 25 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_451]
- Group By Operator [GBY_450] (rows=1 width=12)
+ BROADCAST [RS_466]
+ Group By Operator [GBY_465] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_405]
- Group By Operator [GBY_402] (rows=1 width=12)
+ SHUFFLE [RS_420]
+ Group By Operator [GBY_417] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_399] (rows=8116 width=4)
+ Select Operator [SEL_414] (rows=8116 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_393]
+ Please refer to the previous Select Operator [SEL_408]
<-Map 35 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_456]
+ SHUFFLE [RS_471]
PartitionCols:_col0, _col1
- Select Operator [SEL_455] (rows=14398467 width=221)
+ Select Operator [SEL_470] (rows=14398467 width=221)
Output:["_col0","_col1","_col2","_col3"]
TableScan [TS_78] (rows=14398467 width=221)
default@web_returns,web_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["wr_item_sk","wr_order_number","wr_return_amt","wr_net_loss"]
<-Reducer 7 [CONTAINS] vectorized
- Reduce Output Operator [RS_429]
+ Reduce Output Operator [RS_444]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_428] (rows=59581 width=627)
+ Group By Operator [GBY_443] (rows=59581 width=627)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0, _col1, 0L
- Top N Key Operator [TNK_427] (rows=39721 width=618)
+ Top N Key Operator [TNK_442] (rows=39721 width=618)
keys:_col0, _col1,top n:100
- Select Operator [SEL_426] (rows=822 width=617)
+ Select Operator [SEL_441] (rows=822 width=617)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_425] (rows=822 width=436)
+ Group By Operator [GBY_440] (rows=822 width=436)
Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0
<-Reducer 6 [SIMPLE_EDGE]
SHUFFLE [RS_34]
@@ -466,71 +466,71 @@ Stage-0
Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)"],keys:_col0
Select Operator [SEL_31] (rows=15038783 width=100)
Output:["_col0","_col1","_col2","_col3"]
- Merge Join Operator [MERGEJOIN_363] (rows=15038783 width=100)
- Conds:RS_28._col2=RS_424._col0(Inner),Output:["_col5","_col6","_col9","_col10","_col15"]
+ Merge Join Operator [MERGEJOIN_378] (rows=15038783 width=100)
+ Conds:RS_28._col2=RS_439._col0(Inner),Output:["_col5","_col6","_col9","_col10","_col15"]
<-Map 28 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_424]
+ SHUFFLE [RS_439]
PartitionCols:_col0
- Select Operator [SEL_423] (rows=1704 width=104)
+ Select Operator [SEL_438] (rows=1704 width=104)
Output:["_col0","_col1"]
TableScan [TS_14] (rows=1704 width=104)
default@store,store,Tbl:COMPLETE,Col:COMPLETE,Output:["s_store_sk","s_store_id"]
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_28]
PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_362] (rows=15038783 width=0)
- Conds:RS_25._col3=RS_420._col0(Inner),Output:["_col2","_col5","_col6","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_377] (rows=15038783 width=0)
+ Conds:RS_25._col3=RS_435._col0(Inner),Output:["_col2","_col5","_col6","_col9","_col10"]
<-Map 27 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_420]
+ SHUFFLE [RS_435]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_419]
+ Please refer to the previous Select Operator [SEL_434]
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_25]
PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_361] (rows=30077566 width=57)
- Conds:RS_22._col1=RS_415._col0(Inner),Output:["_col2","_col3","_col5","_col6","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_376] (rows=30077566 width=57)
+ Conds:RS_22._col1=RS_430._col0(Inner),Output:["_col2","_col3","_col5","_col6","_col9","_col10"]
<-Map 26 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_415]
+ SHUFFLE [RS_430]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_414]
+ Please refer to the previous Select Operator [SEL_429]
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_22]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_360] (rows=90232695 width=177)
- Conds:RS_19._col0=RS_394._col0(Inner),Output:["_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_375] (rows=90232695 width=177)
+ Conds:RS_19._col0=RS_409._col0(Inner),Output:["_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
<-Map 12 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_394]
+ SHUFFLE [RS_409]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_393]
+ Please refer to the previous Select Operator [SEL_408]
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_19]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_359] (rows=812149846 width=374)
- Conds:RS_410._col1, _col4=RS_412._col0, _col1(Left Outer),Output:["_col0","_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
+ Merge Join Operator [MERGEJOIN_374] (rows=812149846 width=374)
+ Conds:RS_425._col1, _col4=RS_427._col0, _col1(Left Outer),Output:["_col0","_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_410]
+ SHUFFLE [RS_425]
PartitionCols:_col1, _col4
- Select Operator [SEL_409] (rows=501693263 width=233)
+ Select Operator [SEL_424] (rows=501693263 width=233)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- Filter Operator [FIL_408] (rows=501693263 width=233)
+ Filter Operator [FIL_423] (rows=501693263 width=233)
predicate:(ss_sold_date_sk is not null and ss_promo_sk is not null and ss_store_sk is not null and ss_sold_date_sk BETWEEN DynamicValue(RS_20_date_dim_d_date_sk_min) AND DynamicValue(RS_20_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_20_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_0] (rows=575995635 width=233)
default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_item_sk","ss_store_sk","ss_promo_sk","ss_ticket_number","ss_ext_sales_price","ss_net_profit"]
<-Reducer 13 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_407]
- Group By Operator [GBY_406] (rows=1 width=12)
+ BROADCAST [RS_422]
+ Group By Operator [GBY_421] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_403]
- Group By Operator [GBY_400] (rows=1 width=12)
+ SHUFFLE [RS_418]
+ Group By Operator [GBY_415] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_395] (rows=8116 width=4)
+ Select Operator [SEL_410] (rows=8116 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_393]
+ Please refer to the previous Select Operator [SEL_408]
<-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_412]
+ SHUFFLE [RS_427]
PartitionCols:_col0, _col1
- Select Operator [SEL_411] (rows=57591150 width=224)
+ Select Operator [SEL_426] (rows=57591150 width=224)
Output:["_col0","_col1","_col2","_col3"]
TableScan [TS_3] (rows=57591150 width=224)
default@store_returns,store_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["sr_item_sk","sr_ticket_number","sr_return_amt","sr_net_loss"]
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query85.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query85.q.out
index 199d8c8..d40d4b8 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query85.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query85.q.out
@@ -199,30 +199,30 @@ Stage-0
limit:-1
Stage-1
Reducer 6 vectorized
- File Output Operator [FS_209]
- Limit [LIM_208] (rows=72 width=656)
+ File Output Operator [FS_229]
+ Limit [LIM_228] (rows=72 width=656)
Number of rows:100
- Select Operator [SEL_207] (rows=72 width=656)
+ Select Operator [SEL_227] (rows=72 width=656)
Output:["_col0","_col1","_col2","_col3"]
<-Reducer 5 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_206]
- Select Operator [SEL_205] (rows=72 width=656)
+ SHUFFLE [RS_226]
+ Select Operator [SEL_225] (rows=72 width=656)
Output:["_col4","_col5","_col6","_col7"]
- Top N Key Operator [TNK_204] (rows=72 width=353)
+ Top N Key Operator [TNK_224] (rows=72 width=353)
keys:substr(_col0, 1, 20), (UDFToDouble(_col1) / _col2), CAST( (_col3 / _col4) AS decimal(11,6)), CAST( (_col5 / _col6) AS decimal(11,6)),top n:100
- Group By Operator [GBY_203] (rows=72 width=353)
+ Group By Operator [GBY_223] (rows=72 width=353)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)","sum(VALUE._col2)","count(VALUE._col3)","sum(VALUE._col4)","count(VALUE._col5)"],keys:KEY._col0
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_43]
PartitionCols:_col0
Group By Operator [GBY_42] (rows=288 width=353)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["sum(_col3)","count(_col3)","sum(_col27)","count(_col27)","sum(_col26)","count(_col26)"],keys:_col36
- Merge Join Operator [MERGEJOIN_178] (rows=2912400 width=313)
- Conds:RS_38._col24=RS_202._col0(Inner),Output:["_col3","_col26","_col27","_col36"]
+ Merge Join Operator [MERGEJOIN_198] (rows=2912400 width=313)
+ Conds:RS_38._col24=RS_222._col0(Inner),Output:["_col3","_col26","_col27","_col36"]
<-Map 15 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_202]
+ SHUFFLE [RS_222]
PartitionCols:_col0
- Select Operator [SEL_201] (rows=72 width=101)
+ Select Operator [SEL_221] (rows=72 width=101)
Output:["_col0","_col1"]
TableScan [TS_28] (rows=72 width=101)
default@reason,reason,Tbl:COMPLETE,Col:COMPLETE,Output:["r_reason_sk","r_reason_desc"]
@@ -231,90 +231,90 @@ Stage-0
PartitionCols:_col24
Filter Operator [FIL_36] (rows=2912400 width=280)
predicate:(((_col29 and _col4) or (_col30 and _col5) or (_col31 and _col6)) and ((_col14 and _col15 and _col7) or (_col16 and _col17 and _col8) or (_col18 and _col19 and _col9)))
- Merge Join Operator [MERGEJOIN_177] (rows=10355208 width=280)
+ Merge Join Operator [MERGEJOIN_197] (rows=10355208 width=280)
Conds:RS_33._col1, _col2=RS_34._col9, _col14(Inner),Output:["_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col14","_col15","_col16","_col17","_col18","_col19","_col24","_col26","_col27","_col29","_col30","_col31"]
<-Reducer 12 [SIMPLE_EDGE]
SHUFFLE [RS_34]
PartitionCols:_col9, _col14
Select Operator [SEL_27] (rows=1056644 width=155)
Output:["_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col13","_col14","_col15","_col16","_col18","_col19","_col20"]
- Merge Join Operator [MERGEJOIN_176] (rows=1056644 width=155)
- Conds:RS_24._col1, _col13, _col14=RS_199._col0, _col1, _col2(Inner),Output:["_col0","_col4","_col5","_col6","_col7","_col9","_col10","_col11","_col18","_col19","_col20","_col21","_col22","_col23"]
+ Merge Join Operator [MERGEJOIN_196] (rows=1056644 width=155)
+ Conds:RS_24._col1, _col13, _col14=RS_219._col0, _col1, _col2(Inner),Output:["_col0","_col4","_col5","_col6","_col7","_col9","_col10","_col11","_col18","_col19","_col20","_col21","_col22","_col23"]
<-Map 14 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_199]
+ SHUFFLE [RS_219]
PartitionCols:_col0, _col1, _col2
- Select Operator [SEL_197] (rows=265971 width=207)
+ Select Operator [SEL_217] (rows=265971 width=207)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
- Filter Operator [FIL_196] (rows=265971 width=183)
+ Filter Operator [FIL_216] (rows=265971 width=183)
predicate:((cd_marital_status) IN ('M', 'D', 'U') and (cd_education_status) IN ('4 yr Degree', 'Primary', 'Advanced Degree'))
TableScan [TS_15] (rows=1861800 width=183)
default@customer_demographics,cd1,Tbl:COMPLETE,Col:COMPLETE,Output:["cd_demo_sk","cd_marital_status","cd_education_status"]
<-Reducer 11 [SIMPLE_EDGE]
SHUFFLE [RS_24]
PartitionCols:_col1, _col13, _col14
- Merge Join Operator [MERGEJOIN_175] (rows=1056644 width=312)
- Conds:RS_21._col3=RS_200._col0(Inner),Output:["_col0","_col1","_col4","_col5","_col6","_col7","_col9","_col10","_col11","_col13","_col14"]
+ Merge Join Operator [MERGEJOIN_195] (rows=1056644 width=312)
+ Conds:RS_21._col3=RS_220._col0(Inner),Output:["_col0","_col1","_col4","_col5","_col6","_col7","_col9","_col10","_col11","_col13","_col14"]
<-Map 14 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_200]
+ SHUFFLE [RS_220]
PartitionCols:_col0
- Select Operator [SEL_198] (rows=265971 width=183)
+ Select Operator [SEL_218] (rows=265971 width=183)
Output:["_col0","_col1","_col2"]
- Please refer to the previous Filter Operator [FIL_196]
+ Please refer to the previous Filter Operator [FIL_216]
<-Reducer 10 [SIMPLE_EDGE]
SHUFFLE [RS_21]
PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_174] (rows=1056644 width=135)
- Conds:RS_192._col2=RS_195._col0(Inner),Output:["_col0","_col1","_col3","_col4","_col5","_col6","_col7","_col9","_col10","_col11"]
+ Merge Join Operator [MERGEJOIN_194] (rows=1056644 width=135)
+ Conds:RS_212._col2=RS_215._col0(Inner),Output:["_col0","_col1","_col3","_col4","_col5","_col6","_col7","_col9","_col10","_col11"]
<-Map 13 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_195]
+ SHUFFLE [RS_215]
PartitionCols:_col0
- Select Operator [SEL_194] (rows=3529412 width=16)
+ Select Operator [SEL_214] (rows=3529412 width=16)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_193] (rows=3529412 width=187)
+ Filter Operator [FIL_213] (rows=3529412 width=187)
predicate:((ca_state) IN ('KY', 'GA', 'NM', 'MT', 'OR', 'IN', 'WI', 'MO', 'WV') and (ca_country = 'United States'))
TableScan [TS_9] (rows=40000000 width=187)
default@customer_address,customer_address,Tbl:COMPLETE,Col:COMPLETE,Output:["ca_address_sk","ca_state","ca_country"]
<-Map 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_192]
+ SHUFFLE [RS_212]
PartitionCols:_col2
- Select Operator [SEL_191] (rows=11975292 width=237)
+ Select Operator [SEL_211] (rows=11975292 width=237)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- Filter Operator [FIL_190] (rows=11975292 width=237)
+ Filter Operator [FIL_210] (rows=11975292 width=237)
predicate:(wr_refunded_addr_sk is not null and wr_returning_cdemo_sk is not null and wr_refunded_cdemo_sk is not null and wr_reason_sk is not null)
TableScan [TS_6] (rows=14398467 width=237)
default@web_returns,web_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["wr_item_sk","wr_refunded_cdemo_sk","wr_refunded_addr_sk","wr_returning_cdemo_sk","wr_reason_sk","wr_order_number","wr_fee","wr_refunded_cash"]
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_33]
PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_173] (rows=51392014 width=35)
- Conds:RS_189._col0=RS_181._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
+ Merge Join Operator [MERGEJOIN_193] (rows=51392014 width=35)
+ Conds:RS_209._col0=RS_201._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
<-Map 7 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_181]
+ PARTITION_ONLY_SHUFFLE [RS_201]
PartitionCols:_col0
- Select Operator [SEL_180] (rows=652 width=4)
+ Select Operator [SEL_200] (rows=652 width=4)
Output:["_col0"]
- Filter Operator [FIL_179] (rows=652 width=8)
+ Filter Operator [FIL_199] (rows=652 width=8)
predicate:(d_year = 1998)
TableScan [TS_3] (rows=73049 width=8)
default@date_dim,date_dim,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_year"]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_189]
+ SHUFFLE [RS_209]
PartitionCols:_col0
- Select Operator [SEL_188] (rows=143931136 width=39)
+ Select Operator [SEL_208] (rows=143931136 width=39)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
- Filter Operator [FIL_187] (rows=143931136 width=243)
+ Filter Operator [FIL_207] (rows=143931136 width=243)
predicate:(((ws_sales_price >= 100) or (ws_sales_price <= 150) or (ws_sales_price >= 50) or (ws_sales_price <= 100) or (ws_sales_price >= 150) or (ws_sales_price <= 200)) and ((ws_net_profit >= 100) or (ws_net_profit <= 200) or (ws_net_profit >= 150) or (ws_net_profit <= 300) or (ws_net_profit >= 50) or (ws_net_profit <= 250)) and ws_sold_date_sk is not null and ws_web_page_sk is not null and ws_sold_date_sk BETWEEN DynamicValue(RS_31_date_dim_d_ [...]
TableScan [TS_0] (rows=144002668 width=243)
default@web_sales,web_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ws_sold_date_sk","ws_item_sk","ws_web_page_sk","ws_order_number","ws_quantity","ws_sales_price","ws_net_profit"]
<-Reducer 8 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_186]
- Group By Operator [GBY_185] (rows=1 width=12)
+ BROADCAST [RS_206]
+ Group By Operator [GBY_205] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 7 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_184]
- Group By Operator [GBY_183] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_204]
+ Group By Operator [GBY_203] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_182] (rows=652 width=4)
+ Select Operator [SEL_202] (rows=652 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_180]
+ Please refer to the previous Select Operator [SEL_200]
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query93.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query93.q.out
index 5374a85..4c73d58 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query93.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query93.q.out
@@ -55,16 +55,16 @@ Stage-0
limit:100
Stage-1
Reducer 5 vectorized
- File Output Operator [FS_81]
- Limit [LIM_80] (rows=100 width=112)
+ File Output Operator [FS_91]
+ Limit [LIM_90] (rows=100 width=112)
Number of rows:100
- Select Operator [SEL_79] (rows=38308 width=108)
+ Select Operator [SEL_89] (rows=38308 width=108)
Output:["_col0","_col1"]
<-Reducer 4 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_78]
- Top N Key Operator [TNK_77] (rows=38308 width=112)
+ SHUFFLE [RS_88]
+ Top N Key Operator [TNK_87] (rows=38308 width=112)
keys:_col1, _col0,top n:100
- Group By Operator [GBY_76] (rows=38308 width=112)
+ Group By Operator [GBY_86] (rows=38308 width=112)
Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_17]
@@ -73,49 +73,49 @@ Stage-0
Output:["_col0","_col1"],aggregations:["sum(_col1)"],keys:_col0
Select Operator [SEL_14] (rows=15586502 width=3)
Output:["_col0","_col1"]
- Merge Join Operator [MERGEJOIN_64] (rows=15586502 width=3)
- Conds:RS_11._col0, _col2=RS_75._col0, _col2(Inner),Output:["_col3","_col6","_col8","_col9"]
+ Merge Join Operator [MERGEJOIN_74] (rows=15586502 width=3)
+ Conds:RS_11._col0, _col2=RS_85._col0, _col2(Inner),Output:["_col3","_col6","_col8","_col9"]
<-Reducer 2 [SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_11]
PartitionCols:_col0, _col2
- Merge Join Operator [MERGEJOIN_63] (rows=1522298 width=8)
- Conds:RS_67._col1=RS_70._col0(Inner),Output:["_col0","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_73] (rows=1522298 width=8)
+ Conds:RS_77._col1=RS_80._col0(Inner),Output:["_col0","_col2","_col3"]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_67]
+ SHUFFLE [RS_77]
PartitionCols:_col1
- Select Operator [SEL_66] (rows=55574563 width=15)
+ Select Operator [SEL_76] (rows=55574563 width=15)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_65] (rows=55574563 width=15)
+ Filter Operator [FIL_75] (rows=55574563 width=15)
predicate:sr_reason_sk is not null
TableScan [TS_0] (rows=57591150 width=15)
default@store_returns,store_returns,Tbl:COMPLETE,Col:COMPLETE,Output:["sr_item_sk","sr_reason_sk","sr_ticket_number","sr_return_quantity"]
<-Map 7 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_70]
+ SHUFFLE [RS_80]
PartitionCols:_col0
- Select Operator [SEL_69] (rows=1 width=4)
+ Select Operator [SEL_79] (rows=1 width=4)
Output:["_col0"]
- Filter Operator [FIL_68] (rows=1 width=101)
+ Filter Operator [FIL_78] (rows=1 width=101)
predicate:(r_reason_desc = 'Did not like the warranty')
TableScan [TS_3] (rows=72 width=101)
default@reason,reason,Tbl:COMPLETE,Col:COMPLETE,Output:["r_reason_sk","r_reason_desc"]
<-Map 8 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_75]
+ SHUFFLE [RS_85]
PartitionCols:_col0, _col2
- Select Operator [SEL_74] (rows=575995635 width=122)
+ Select Operator [SEL_84] (rows=575995635 width=122)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_73] (rows=575995635 width=122)
- predicate:(ss_ticket_number BETWEEN DynamicValue(RS_11_store_returns_sr_ticket_number_min) AND DynamicValue(RS_11_store_returns_sr_ticket_number_max) and in_bloom_filter(ss_ticket_number, DynamicValue(RS_11_store_returns_sr_ticket_number_bloom_filter)))
+ Filter Operator [FIL_83] (rows=575995635 width=122)
+ predicate:(ss_item_sk BETWEEN DynamicValue(RS[72]_col0) AND DynamicValue(RS[72]_col1) and ss_ticket_number BETWEEN DynamicValue(RS[72]_col2) AND DynamicValue(RS[72]_col3) and in_bloom_filter(hash(ss_item_sk,ss_ticket_number), DynamicValue(RS[72]_col4)))
TableScan [TS_6] (rows=575995635 width=122)
default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_item_sk","ss_customer_sk","ss_ticket_number","ss_quantity","ss_sales_price"]
<-Reducer 6 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_72]
- Group By Operator [GBY_71] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ BROADCAST [RS_82]
+ Group By Operator [GBY_81] (rows=1 width=160)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","min(VALUE._col2)","max(VALUE._col3)","bloom_filter(VALUE._col4, expectedEntries=1000000)"]
<-Reducer 2 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_60]
- Group By Operator [GBY_59] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_58] (rows=1522298 width=8)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_63]
+ PARTITION_ONLY_SHUFFLE [RS_70]
+ Group By Operator [GBY_69] (rows=1 width=160)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["min(_col0)","max(_col0)","min(_col1)","max(_col1)","bloom_filter(_col3, expectedEntries=1000000)"]
+ Select Operator [SEL_68] (rows=1522298 width=12)
+ Output:["_col0","_col1","_col3"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_73]
diff --git a/ql/src/test/results/clientpositive/perf/tez/query14.q.out b/ql/src/test/results/clientpositive/perf/tez/query14.q.out
index 1d891b0..c35c54b 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query14.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query14.q.out
@@ -1,6 +1,6 @@
-Warning: Shuffle Join MERGEJOIN[1178][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 6' is a cross product
-Warning: Shuffle Join MERGEJOIN[1185][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 16' is a cross product
-Warning: Shuffle Join MERGEJOIN[1192][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 22' is a cross product
+Warning: Shuffle Join MERGEJOIN[1238][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 6' is a cross product
+Warning: Shuffle Join MERGEJOIN[1245][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 16' is a cross product
+Warning: Shuffle Join MERGEJOIN[1252][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 22' is a cross product
PREHOOK: query: explain
with cross_items as
(select i_item_sk ss_item_sk
@@ -292,36 +292,36 @@ Stage-0
limit:100
Stage-1
Reducer 9 vectorized
- File Output Operator [FS_1365]
- Limit [LIM_1364] (rows=100 width=223)
+ File Output Operator [FS_1425]
+ Limit [LIM_1424] (rows=100 width=223)
Number of rows:100
- Select Operator [SEL_1363] (rows=304320 width=222)
+ Select Operator [SEL_1423] (rows=304320 width=222)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
<-Reducer 8 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1362]
- Select Operator [SEL_1361] (rows=304320 width=222)
+ SHUFFLE [RS_1422]
+ Select Operator [SEL_1421] (rows=304320 width=222)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Group By Operator [GBY_1360] (rows=304320 width=230)
+ Group By Operator [GBY_1420] (rows=304320 width=230)
Output:["_col0","_col1","_col2","_col3","_col5","_col6"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4
<-Union 7 [SIMPLE_EDGE]
<-Reducer 16 [CONTAINS]
- Reduce Output Operator [RS_1191]
+ Reduce Output Operator [RS_1251]
PartitionCols:_col0, _col1, _col2, _col3, _col4
- Group By Operator [GBY_1190] (rows=304320 width=230)
+ Group By Operator [GBY_1250] (rows=304320 width=230)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["sum(_col4)","sum(_col5)"],keys:_col0, _col1, _col2, _col3, 0L
- Top N Key Operator [TNK_1189] (rows=121728 width=220)
+ Top N Key Operator [TNK_1249] (rows=121728 width=220)
keys:_col0, _col1, _col2, _col3,top n:100
- Select Operator [SEL_1187] (rows=40576 width=222)
+ Select Operator [SEL_1247] (rows=40576 width=222)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_1186] (rows=40576 width=243)
+ Filter Operator [FIL_1246] (rows=40576 width=243)
predicate:(_col3 > _col5)
- Merge Join Operator [MERGEJOIN_1185] (rows=121728 width=243)
+ Merge Join Operator [MERGEJOIN_1245] (rows=121728 width=243)
Conds:(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
<-Reducer 15 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1377]
- Filter Operator [FIL_1376] (rows=121728 width=131)
+ PARTITION_ONLY_SHUFFLE [RS_1437]
+ Filter Operator [FIL_1436] (rows=121728 width=131)
predicate:_col3 is not null
- Group By Operator [GBY_1375] (rows=121728 width=131)
+ Group By Operator [GBY_1435] (rows=121728 width=131)
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 14 [SIMPLE_EDGE]
SHUFFLE [RS_244]
@@ -330,354 +330,354 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col3)","count()"],keys:_col0, _col1, _col2
Select Operator [SEL_241] (rows=7790806 width=106)
Output:["_col0","_col1","_col2","_col3"]
- Merge Join Operator [MERGEJOIN_1173] (rows=7790806 width=106)
+ Merge Join Operator [MERGEJOIN_1233] (rows=7790806 width=106)
Conds:RS_238._col1=RS_239._col0(Left Semi),Output:["_col2","_col3","_col6","_col7","_col8"]
<-Reducer 13 [SIMPLE_EDGE]
SHUFFLE [RS_238]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_1149] (rows=7790806 width=110)
- Conds:RS_233._col1=RS_1345._col0(Inner),Output:["_col1","_col2","_col3","_col6","_col7","_col8"]
+ Merge Join Operator [MERGEJOIN_1209] (rows=7790806 width=110)
+ Conds:RS_233._col1=RS_1405._col0(Inner),Output:["_col1","_col2","_col3","_col6","_col7","_col8"]
<-Map 59 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1345]
+ SHUFFLE [RS_1405]
PartitionCols:_col0
- Select Operator [SEL_1336] (rows=462000 width=15)
+ Select Operator [SEL_1396] (rows=462000 width=15)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_1327] (rows=462000 width=15)
+ Filter Operator [FIL_1387] (rows=462000 width=15)
predicate:i_item_sk is not null
TableScan [TS_6] (rows=462000 width=15)
default@item,item,Tbl:COMPLETE,Col:COMPLETE,Output:["i_item_sk","i_brand_id","i_class_id","i_category_id"]
<-Reducer 12 [SIMPLE_EDGE]
SHUFFLE [RS_233]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_1148] (rows=7790806 width=98)
- Conds:RS_1370._col0=RS_1295._col0(Inner),Output:["_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_1208] (rows=7790806 width=98)
+ Conds:RS_1430._col0=RS_1355._col0(Inner),Output:["_col1","_col2","_col3"]
<-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1295]
+ SHUFFLE [RS_1355]
PartitionCols:_col0
- Select Operator [SEL_1291] (rows=50 width=4)
+ Select Operator [SEL_1351] (rows=50 width=4)
Output:["_col0"]
- Filter Operator [FIL_1289] (rows=50 width=12)
+ Filter Operator [FIL_1349] (rows=50 width=12)
predicate:((d_year = 2000) and (d_moy = 11) and d_date_sk is not null)
TableScan [TS_3] (rows=73049 width=12)
default@date_dim,date_dim,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_year","d_moy"]
<-Map 75 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1370]
+ SHUFFLE [RS_1430]
PartitionCols:_col0
- Select Operator [SEL_1369] (rows=286549727 width=123)
+ Select Operator [SEL_1429] (rows=286549727 width=123)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_1368] (rows=286549727 width=123)
+ Filter Operator [FIL_1428] (rows=286549727 width=123)
predicate:(cs_sold_date_sk is not null and cs_item_sk is not null and cs_sold_date_sk BETWEEN DynamicValue(RS_231_date_dim_d_date_sk_min) AND DynamicValue(RS_231_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_231_date_dim_d_date_sk_bloom_filter)))
TableScan [TS_146] (rows=287989836 width=123)
default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["cs_sold_date_sk","cs_item_sk","cs_quantity","cs_list_price"]
<-Reducer 17 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_1367]
- Group By Operator [GBY_1366] (rows=1 width=12)
+ BROADCAST [RS_1427]
+ Group By Operator [GBY_1426] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1313]
- Group By Operator [GBY_1308] (rows=1 width=12)
+ SHUFFLE [RS_1373]
+ Group By Operator [GBY_1368] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_1296] (rows=50 width=4)
+ Select Operator [SEL_1356] (rows=50 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_1291]
+ Please refer to the previous Select Operator [SEL_1351]
<-Reducer 33 [SIMPLE_EDGE]
SHUFFLE [RS_239]
PartitionCols:_col0
Group By Operator [GBY_237] (rows=362 width=4)
Output:["_col0"],keys:_col0
- Merge Join Operator [MERGEJOIN_1156] (rows=724 width=4)
- Conds:RS_1346._col1, _col2, _col3=RS_1374._col0, _col1, _col2(Inner),Output:["_col0"]
+ Merge Join Operator [MERGEJOIN_1216] (rows=724 width=4)
+ Conds:RS_1406._col1, _col2, _col3=RS_1434._col0, _col1, _col2(Inner),Output:["_col0"]
<-Map 59 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1346]
+ SHUFFLE [RS_1406]
PartitionCols:_col1, _col2, _col3
- Select Operator [SEL_1337] (rows=458612 width=15)
+ Select Operator [SEL_1397] (rows=458612 width=15)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_1328] (rows=458612 width=15)
+ Filter Operator [FIL_1388] (rows=458612 width=15)
predicate:(i_category_id is not null and i_brand_id is not null and i_class_id is not null and i_item_sk is not null)
Please refer to the previous TableScan [TS_6]
<-Reducer 32 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1374]
+ SHUFFLE [RS_1434]
PartitionCols:_col0, _col1, _col2
- Select Operator [SEL_1373] (rows=1 width=12)
+ Select Operator [SEL_1433] (rows=1 width=12)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_1372] (rows=1 width=20)
+ Filter Operator [FIL_1432] (rows=1 width=20)
predicate:(_col3 = 3L)
- Group By Operator [GBY_1371] (rows=121728 width=19)
+ Group By Operator [GBY_1431] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Union 31 [SIMPLE_EDGE]
<-Reducer 30 [CONTAINS] vectorized
- Reduce Output Operator [RS_1411]
+ Reduce Output Operator [RS_1471]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_1410] (rows=121728 width=19)
+ Group By Operator [GBY_1470] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(_col3)"],keys:_col0, _col1, _col2
- Group By Operator [GBY_1409] (rows=121728 width=19)
+ Group By Operator [GBY_1469] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 25 [SIMPLE_EDGE]
SHUFFLE [RS_175]
PartitionCols:_col0, _col1, _col2
Group By Operator [GBY_28] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count()"],keys:_col4, _col5, _col6
- Merge Join Operator [MERGEJOIN_1139] (rows=14628613 width=11)
- Conds:RS_24._col1=RS_1342._col0(Inner),Output:["_col4","_col5","_col6"]
+ Merge Join Operator [MERGEJOIN_1199] (rows=14628613 width=11)
+ Conds:RS_24._col1=RS_1402._col0(Inner),Output:["_col4","_col5","_col6"]
<-Map 59 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1342]
+ SHUFFLE [RS_1402]
PartitionCols:_col0
- Select Operator [SEL_1333] (rows=458612 width=15)
+ Select Operator [SEL_1393] (rows=458612 width=15)
Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_1324] (rows=458612 width=15)
+ Filter Operator [FIL_1384] (rows=458612 width=15)
predicate:(i_category_id is not null and i_brand_id is not null and i_class_id is not null and i_item_sk is not null)
Please refer to the previous TableScan [TS_6]
<-Reducer 24 [SIMPLE_EDGE]
SHUFFLE [RS_24]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_1138] (rows=14736682 width=4)
- Conds:RS_1402._col0=RS_1299._col0(Inner),Output:["_col1"]
+ Merge Join Operator [MERGEJOIN_1198] (rows=14736682 width=4)
+ Conds:RS_1462._col0=RS_1359._col0(Inner),Output:["_col1"]
<-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1299]
+ SHUFFLE [RS_1359]
PartitionCols:_col0
- Select Operator [SEL_1292] (rows=1957 width=4)
+ Select Operator [SEL_1352] (rows=1957 width=4)
Output:["_col0"]
- Filter Operator [FIL_1290] (rows=1957 width=8)
+ Filter Operator [FIL_1350] (rows=1957 width=8)
predicate:(d_year BETWEEN 1999 AND 2001 and d_date_sk is not null)
Please refer to the previous TableScan [TS_3]
<-Map 60 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1402]
+ SHUFFLE [RS_1462]
PartitionCols:_col0
- Select Operator [SEL_1400] (rows=550076554 width=7)
+ Select Operator [SEL_1460] (rows=550076554 width=7)
Output:["_col0","_col1"]
- Filter Operator [FIL_1398] (rows=550076554 width=7)
+ Filter Operator [FIL_1458] (rows=550076554 width=7)
predicate:(ss_sold_date_sk is not null and ss_item_sk is not null)
TableScan [TS_12] (rows=575995635 width=7)
default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_item_sk","ss_quantity","ss_list_price"]
<-Reducer 41 [CONTAINS] vectorized
- Reduce Output Operator [RS_1425]
+ Reduce Output Operator [RS_1485]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_1424] (rows=121728 width=19)
+ Group By Operator [GBY_1484] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(_col3)"],keys:_col0, _col1, _col2
- Group By Operator [GBY_1423] (rows=121728 width=19)
+ Group By Operator [GBY_1483] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 39 [SIMPLE_EDGE]
SHUFFLE [RS_195]
PartitionCols:_col0, _col1, _col2
Group By Operator [GBY_48] (rows=121728 width=19)
Output:["_col0","_col1","_col2","_col3"],aggregations:["count()"],keys:_col4, _col5, _col6
... 5908 lines suppressed ...