You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by za...@apache.org on 2024/10/10 11:50:03 UTC
(hive) branch master updated: HIVE-28259: Common table expression detection and rewrites using CBO (Stamatis Zampetakis reviewed by Alessandro Solimando, Aman Sinha)
This is an automated email from the ASF dual-hosted git repository.
zabetak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new b6c191db42c HIVE-28259: Common table expression detection and rewrites using CBO (Stamatis Zampetakis reviewed by Alessandro Solimando, Aman Sinha)
b6c191db42c is described below
commit b6c191db42c7bc9911ce566e5f28b38dde591718
Author: Stamatis Zampetakis <za...@gmail.com>
AuthorDate: Mon May 13 17:21:05 2024 +0200
HIVE-28259: Common table expression detection and rewrites using CBO (Stamatis Zampetakis reviewed by Alessandro Solimando, Aman Sinha)
1. Add `applyCteRewriting` phase in `CalcitePlanner` for detecting and using CTEs; ensure rewrite logic is consistent with existing `hive.optimize.cte.materialize.*` properties.
2. Model CTEs as materialized views (MVs) and add utility method in `HiveMaterializedViewUtils` for mapping a CTE to a `RelOptMaterialization`.
3. Refactor core MV rewrite logic in `CalcitePlanner` to use during CTE rewrite and exploit CTEs in a cost-based manner.
4. Add `HiveTableSpool` operator to represent CTEs and handle them in the plan using new rules: `TableScanToSpoolRule` and `RemoveUnusedCteRule`.
5. Add `TableScanToSpoolRule`, and `RemoveInfrequentCteRule` to add/remove spools from the plan.
6. Enhance/Enrich metadata handlers for handling the Spool operator.
7. Add `AggregatedColumns` metadata (and respective handler and metadata query), for controlling if a CTE is a "full aggregate" at the CBO (RelNode) level to ensure consistent behavior with `hive.optimize.cte.materialize.full.aggregate.only` property.
8. Add `HiveSqlTypeUtil.containsSqlType` for detecting and skipping the creation of CTEs with untyped nulls since they are not supported (HIVE-11217).
9. Add `hive.optimize.cte.suggester.class` and CommonTableExpressionSuggester interface to provide pluggable CTE detection logic. Given that CTE detection logic can range from basic tree traversal algorithms to complex workload analysis frameworks this part needs to be configurable since there is no one-size-fits-all implementation. The configuration property also allows proprietary algorithms to be integrated in HiveServer2 by implementing the necessary APIs and adding the jars in th [...]
10. Add prototype implementation for CTE detection logic in CommonTableExpressionIdentitySuggester using CommonRelSubExprRegisterRule and CommonTableExpressionRegistry. Although the implementation is rather simple it can discover various interesting CTEs as demonstrated by the tests and can be indeed useful in a prod environment.
11. Map spool(s) to `WITH` clauses during the RelNode to AST conversion (ASTConverter, ASTBuilder, PlanModifierForASTConv) to exploit existing CTE materialization feature (HIVE-11752).
12. Modify (slighly) `SemanticAnalyzer`/`CalcitePlanner` to enable AST-based CTE materialization (getMetadata) post CBO run.
13. Tests for:
* CTE detection logic using the `CommonTableExpressionPrintSuggester` (`TestTezTPCDS30TBPerfCliDriver`); use only in cbo_query* tests to avoid redundancy.
* demonstrating (end-to-end) the CTE feature (cte_cbo_rewrite_0.q)
* verify coherence of CTE rewrite with `hive.optimize.cte.materialize.full.aggregate.only` (cte_mat_12.q)
* spool JSON serialization (cte_cbo_plan_json.q)
Close apache/hive#5249
---
.../java/org/apache/hadoop/hive/conf/HiveConf.java | 4 +
.../ql/optimizer/calcite/CalciteCteException.java | 23 +
.../CommonTableExpressionIdentitySuggester.java | 54 ++
.../CommonTableExpressionPrintSuggester.java | 44 +
.../calcite/CommonTableExpressionRegistry.java | 54 ++
.../calcite/CommonTableExpressionSuggester.java | 48 ++
.../CommonTableExpressionSuggesterFactory.java | 48 ++
.../hive/ql/optimizer/calcite/HiveCalciteUtil.java | 17 +
.../calcite/HiveDefaultRelMetadataProvider.java | 6 +
.../ql/optimizer/calcite/HiveRelFactories.java | 6 +-
.../hive/ql/optimizer/calcite/HiveSqlTypeUtil.java | 49 ++
.../hive/ql/optimizer/calcite/RelOptHiveTable.java | 4 +
.../calcite/reloperators/HiveTableSpool.java | 41 +
.../rules/CommonRelSubExprRegisterRule.java | 84 ++
.../ql/optimizer/calcite/rules/CteRuleConfig.java | 41 +
.../calcite/rules/HivePartitionPruneRule.java | 3 +
.../calcite/rules/RemoveInfrequentCteRule.java | 54 ++
.../calcite/rules/TableScanToSpoolRule.java | 67 ++
.../rules/views/HiveMaterializedViewUtils.java | 39 +
.../optimizer/calcite/stats/AggregatedColumns.java | 50 ++
.../calcite/stats/HiveRelMdAggregatedColumns.java | 81 ++
.../calcite/stats/HiveRelMdColumnUniqueness.java | 5 +
.../calcite/stats/HiveRelMdDistinctRowCount.java | 6 +
.../calcite/stats/HiveRelMdTableReferences.java | 45 ++
.../calcite/stats/HiveRelMetadataQuery.java | 52 ++
.../ql/optimizer/calcite/stats/MetadataMethod.java | 36 +
.../optimizer/calcite/translator/ASTBuilder.java | 12 +
.../optimizer/calcite/translator/ASTConverter.java | 30 +-
.../calcite/translator/PlanModifierForASTConv.java | 8 +
.../hadoop/hive/ql/parse/CalcitePlanner.java | 164 +++-
.../hadoop/hive/ql/parse/SemanticAnalyzer.java | 7 +-
.../queries/clientpositive/cte_cbo_plan_json.q | 16 +
.../queries/clientpositive/cte_cbo_rewrite_0.q | 161 ++++
ql/src/test/queries/clientpositive/cte_mat_12.q | 45 ++
.../test/queries/clientpositive/perf/cbo_query1.q | 1 +
.../test/queries/clientpositive/perf/cbo_query10.q | 1 +
.../test/queries/clientpositive/perf/cbo_query11.q | 1 +
.../test/queries/clientpositive/perf/cbo_query12.q | 1 +
.../test/queries/clientpositive/perf/cbo_query13.q | 1 +
.../test/queries/clientpositive/perf/cbo_query14.q | 1 +
.../test/queries/clientpositive/perf/cbo_query15.q | 1 +
.../test/queries/clientpositive/perf/cbo_query16.q | 1 +
.../test/queries/clientpositive/perf/cbo_query17.q | 1 +
.../test/queries/clientpositive/perf/cbo_query18.q | 1 +
.../test/queries/clientpositive/perf/cbo_query19.q | 1 +
.../test/queries/clientpositive/perf/cbo_query2.q | 1 +
.../test/queries/clientpositive/perf/cbo_query20.q | 1 +
.../test/queries/clientpositive/perf/cbo_query21.q | 1 +
.../test/queries/clientpositive/perf/cbo_query22.q | 1 +
.../test/queries/clientpositive/perf/cbo_query23.q | 1 +
.../test/queries/clientpositive/perf/cbo_query24.q | 1 +
.../test/queries/clientpositive/perf/cbo_query25.q | 1 +
.../test/queries/clientpositive/perf/cbo_query26.q | 1 +
.../test/queries/clientpositive/perf/cbo_query27.q | 1 +
.../test/queries/clientpositive/perf/cbo_query28.q | 1 +
.../test/queries/clientpositive/perf/cbo_query29.q | 1 +
.../test/queries/clientpositive/perf/cbo_query3.q | 1 +
.../test/queries/clientpositive/perf/cbo_query30.q | 1 +
.../test/queries/clientpositive/perf/cbo_query31.q | 1 +
.../test/queries/clientpositive/perf/cbo_query32.q | 1 +
.../test/queries/clientpositive/perf/cbo_query33.q | 1 +
.../test/queries/clientpositive/perf/cbo_query34.q | 1 +
.../test/queries/clientpositive/perf/cbo_query35.q | 1 +
.../test/queries/clientpositive/perf/cbo_query36.q | 1 +
.../test/queries/clientpositive/perf/cbo_query37.q | 1 +
.../test/queries/clientpositive/perf/cbo_query38.q | 1 +
.../test/queries/clientpositive/perf/cbo_query39.q | 1 +
.../test/queries/clientpositive/perf/cbo_query4.q | 1 +
.../test/queries/clientpositive/perf/cbo_query40.q | 1 +
.../test/queries/clientpositive/perf/cbo_query41.q | 1 +
.../test/queries/clientpositive/perf/cbo_query42.q | 1 +
.../test/queries/clientpositive/perf/cbo_query43.q | 1 +
.../test/queries/clientpositive/perf/cbo_query44.q | 1 +
.../test/queries/clientpositive/perf/cbo_query45.q | 1 +
.../test/queries/clientpositive/perf/cbo_query46.q | 1 +
.../test/queries/clientpositive/perf/cbo_query47.q | 1 +
.../test/queries/clientpositive/perf/cbo_query48.q | 1 +
.../test/queries/clientpositive/perf/cbo_query49.q | 1 +
.../test/queries/clientpositive/perf/cbo_query5.q | 1 +
.../test/queries/clientpositive/perf/cbo_query50.q | 1 +
.../test/queries/clientpositive/perf/cbo_query51.q | 1 +
.../test/queries/clientpositive/perf/cbo_query52.q | 1 +
.../test/queries/clientpositive/perf/cbo_query53.q | 1 +
.../test/queries/clientpositive/perf/cbo_query54.q | 1 +
.../test/queries/clientpositive/perf/cbo_query55.q | 1 +
.../test/queries/clientpositive/perf/cbo_query56.q | 1 +
.../test/queries/clientpositive/perf/cbo_query57.q | 1 +
.../test/queries/clientpositive/perf/cbo_query58.q | 1 +
.../test/queries/clientpositive/perf/cbo_query59.q | 1 +
.../test/queries/clientpositive/perf/cbo_query6.q | 1 +
.../test/queries/clientpositive/perf/cbo_query60.q | 1 +
.../test/queries/clientpositive/perf/cbo_query61.q | 1 +
.../test/queries/clientpositive/perf/cbo_query62.q | 1 +
.../test/queries/clientpositive/perf/cbo_query63.q | 1 +
.../test/queries/clientpositive/perf/cbo_query64.q | 1 +
.../test/queries/clientpositive/perf/cbo_query65.q | 1 +
.../test/queries/clientpositive/perf/cbo_query66.q | 1 +
.../test/queries/clientpositive/perf/cbo_query67.q | 1 +
.../test/queries/clientpositive/perf/cbo_query68.q | 1 +
.../test/queries/clientpositive/perf/cbo_query69.q | 1 +
.../test/queries/clientpositive/perf/cbo_query7.q | 1 +
.../test/queries/clientpositive/perf/cbo_query70.q | 1 +
.../test/queries/clientpositive/perf/cbo_query71.q | 1 +
.../test/queries/clientpositive/perf/cbo_query72.q | 1 +
.../test/queries/clientpositive/perf/cbo_query73.q | 1 +
.../test/queries/clientpositive/perf/cbo_query74.q | 1 +
.../test/queries/clientpositive/perf/cbo_query75.q | 1 +
.../test/queries/clientpositive/perf/cbo_query76.q | 1 +
.../test/queries/clientpositive/perf/cbo_query77.q | 1 +
.../test/queries/clientpositive/perf/cbo_query78.q | 1 +
.../test/queries/clientpositive/perf/cbo_query79.q | 1 +
.../test/queries/clientpositive/perf/cbo_query8.q | 1 +
.../test/queries/clientpositive/perf/cbo_query80.q | 1 +
.../test/queries/clientpositive/perf/cbo_query81.q | 1 +
.../test/queries/clientpositive/perf/cbo_query82.q | 1 +
.../test/queries/clientpositive/perf/cbo_query83.q | 1 +
.../test/queries/clientpositive/perf/cbo_query84.q | 1 +
.../test/queries/clientpositive/perf/cbo_query85.q | 1 +
.../test/queries/clientpositive/perf/cbo_query86.q | 1 +
.../test/queries/clientpositive/perf/cbo_query87.q | 1 +
.../test/queries/clientpositive/perf/cbo_query88.q | 1 +
.../test/queries/clientpositive/perf/cbo_query89.q | 1 +
.../test/queries/clientpositive/perf/cbo_query9.q | 1 +
.../test/queries/clientpositive/perf/cbo_query90.q | 1 +
.../test/queries/clientpositive/perf/cbo_query91.q | 1 +
.../test/queries/clientpositive/perf/cbo_query92.q | 1 +
.../test/queries/clientpositive/perf/cbo_query93.q | 1 +
.../test/queries/clientpositive/perf/cbo_query94.q | 1 +
.../test/queries/clientpositive/perf/cbo_query95.q | 1 +
.../test/queries/clientpositive/perf/cbo_query96.q | 1 +
.../test/queries/clientpositive/perf/cbo_query97.q | 1 +
.../test/queries/clientpositive/perf/cbo_query98.q | 1 +
.../test/queries/clientpositive/perf/cbo_query99.q | 1 +
.../clientpositive/llap/cte_cbo_plan_json.q.out | 37 +
.../clientpositive/llap/cte_cbo_rewrite_0.q.out | 883 +++++++++++++++++++++
.../results/clientpositive/llap/cte_mat_12.q.out | 182 +++++
.../llap/partition_explain_ddl.q.out | 72 +-
.../perf/tpcds30tb/tez/cbo_query1.q.out | 5 +
.../perf/tpcds30tb/tez/cbo_query10.q.out | 5 +
.../perf/tpcds30tb/tez/cbo_query11.q.out | 24 +
.../perf/tpcds30tb/tez/cbo_query2.q.out | 15 +
.../perf/tpcds30tb/tez/cbo_query23.q.out | 41 +
.../perf/tpcds30tb/tez/cbo_query24.q.out | 20 +
.../perf/tpcds30tb/tez/cbo_query30.q.out | 10 +
.../perf/tpcds30tb/tez/cbo_query31.q.out | 30 +
.../perf/tpcds30tb/tez/cbo_query32.q.out | 5 +
.../perf/tpcds30tb/tez/cbo_query33.q.out | 20 +
.../perf/tpcds30tb/tez/cbo_query35.q.out | 5 +
.../perf/tpcds30tb/tez/cbo_query38.q.out | 5 +
.../perf/tpcds30tb/tez/cbo_query4.q.out | 30 +
.../perf/tpcds30tb/tez/cbo_query44.q.out | 14 +
.../perf/tpcds30tb/tez/cbo_query47.q.out | 38 +
.../perf/tpcds30tb/tez/cbo_query49.q.out | 5 +
.../perf/tpcds30tb/tez/cbo_query5.q.out | 5 +
.../perf/tpcds30tb/tez/cbo_query51.q.out | 5 +
.../perf/tpcds30tb/tez/cbo_query54.q.out | 8 +
.../perf/tpcds30tb/tez/cbo_query56.q.out | 15 +
.../perf/tpcds30tb/tez/cbo_query57.q.out | 38 +
.../perf/tpcds30tb/tez/cbo_query58.q.out | 20 +
.../perf/tpcds30tb/tez/cbo_query59.q.out | 10 +
.../perf/tpcds30tb/tez/cbo_query60.q.out | 15 +
.../perf/tpcds30tb/tez/cbo_query61.q.out | 24 +
.../perf/tpcds30tb/tez/cbo_query64.q.out | 39 +
.../perf/tpcds30tb/tez/cbo_query65.q.out | 10 +
.../perf/tpcds30tb/tez/cbo_query66.q.out | 15 +
.../perf/tpcds30tb/tez/cbo_query69.q.out | 5 +
.../perf/tpcds30tb/tez/cbo_query71.q.out | 5 +
.../perf/tpcds30tb/tez/cbo_query74.q.out | 20 +
.../perf/tpcds30tb/tez/cbo_query75.q.out | 30 +
.../perf/tpcds30tb/tez/cbo_query77.q.out | 5 +
.../perf/tpcds30tb/tez/cbo_query78.q.out | 5 +
.../perf/tpcds30tb/tez/cbo_query80.q.out | 15 +
.../perf/tpcds30tb/tez/cbo_query81.q.out | 10 +
.../perf/tpcds30tb/tez/cbo_query83.q.out | 15 +
.../perf/tpcds30tb/tez/cbo_query87.q.out | 5 +
.../perf/tpcds30tb/tez/cbo_query88.q.out | 15 +
.../perf/tpcds30tb/tez/cbo_query90.q.out | 14 +
.../perf/tpcds30tb/tez/cbo_query92.q.out | 5 +
.../perf/tpcds30tb/tez/cbo_query97.q.out | 5 +
179 files changed, 3226 insertions(+), 81 deletions(-)
diff --git a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
index 635d54415a4..b6bb8ea3d55 100644
--- a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
+++ b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
@@ -28,6 +28,7 @@ import org.apache.hadoop.hive.common.FileUtils;
import org.apache.hadoop.hive.common.ZooKeeperHiveHelper;
import org.apache.hadoop.hive.common.classification.InterfaceAudience;
import org.apache.hadoop.hive.common.classification.InterfaceAudience.LimitedPrivate;
+import org.apache.hadoop.hive.common.classification.InterfaceStability;
import org.apache.hadoop.hive.common.type.TimestampTZUtil;
import org.apache.hadoop.hive.conf.Validator.PatternSet;
import org.apache.hadoop.hive.conf.Validator.RangeValidator;
@@ -2763,6 +2764,9 @@ public class HiveConf extends Configuration {
+ " provides an optimization if it is accurate."),
// CTE
+ @InterfaceStability.Unstable
+ HIVE_CTE_SUGGESTER_CLASS("hive.optimize.cte.suggester.class", "",
+ "Class for finding and suggesting common table expressions (CTEs) based on a given query. The class must implement the CommonTableExpressionSuggester interface."),
HIVE_CTE_MATERIALIZE_THRESHOLD("hive.optimize.cte.materialize.threshold", 3,
"If the number of references to a CTE clause exceeds this threshold, Hive will materialize it\n" +
"before executing the main query block. -1 will disable this feature."),
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CalciteCteException.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CalciteCteException.java
new file mode 100644
index 00000000000..dd5499b2f6b
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CalciteCteException.java
@@ -0,0 +1,23 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite;
+
+public class CalciteCteException extends RuntimeException {
+ public CalciteCteException(final String message, final Throwable cause) {
+ super(message, cause);
+ }
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CommonTableExpressionIdentitySuggester.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CommonTableExpressionIdentitySuggester.java
new file mode 100644
index 00000000000..7713105eb5f
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CommonTableExpressionIdentitySuggester.java
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite;
+
+import org.apache.calcite.plan.Contexts;
+import org.apache.calcite.plan.hep.HepPlanner;
+import org.apache.calcite.plan.hep.HepProgram;
+import org.apache.calcite.plan.hep.HepProgramBuilder;
+import org.apache.calcite.rel.RelNode;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hive.common.classification.InterfaceAudience;
+import org.apache.hadoop.hive.common.classification.InterfaceStability;
+import org.apache.hadoop.hive.ql.optimizer.calcite.rules.CommonRelSubExprRegisterRule;
+
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Suggester for common table expressions that appear as is (identical trees) more than once in the query plan.
+ */
+@InterfaceAudience.Public
+@InterfaceStability.Unstable
+public class CommonTableExpressionIdentitySuggester implements CommonTableExpressionSuggester {
+
+ @Override
+ public List<RelNode> suggest(final RelNode input, final Configuration configuration) {
+ CommonTableExpressionRegistry localRegistry = new CommonTableExpressionRegistry();
+ HepProgram ruleProgram = new HepProgramBuilder()
+ .addRuleInstance(CommonRelSubExprRegisterRule.JOIN)
+ .addRuleInstance(CommonRelSubExprRegisterRule.AGGREGATE)
+ .addRuleInstance(CommonRelSubExprRegisterRule.FILTER)
+ .addRuleInstance(CommonRelSubExprRegisterRule.PROJECT)
+ .build();
+ HepPlanner planner = new HepPlanner(ruleProgram, Contexts.of(localRegistry));
+ planner.setRoot(input);
+ planner.findBestExp();
+ return localRegistry.entries().collect(Collectors.toList());
+ }
+
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CommonTableExpressionPrintSuggester.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CommonTableExpressionPrintSuggester.java
new file mode 100644
index 00000000000..a83f3413203
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CommonTableExpressionPrintSuggester.java
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite;
+
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.rel.RelNode;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hive.common.classification.InterfaceAudience;
+import org.apache.hadoop.hive.ql.session.SessionState;
+
+import java.util.List;
+
+/**
+ * Suggester for printing common table expressions to the Session's console.
+ */
+@InterfaceAudience.Private
+public class CommonTableExpressionPrintSuggester implements CommonTableExpressionSuggester {
+
+ private final CommonTableExpressionSuggester internal = new CommonTableExpressionIdentitySuggester();
+
+ @Override
+ public List<RelNode> suggest(final RelNode input, final Configuration configuration) {
+ List<RelNode> result = internal.suggest(input, configuration);
+ // Ensure CTEs are printed deterministically to avoid test flakiness
+ result.stream().map(RelOptUtil::toString).sorted()
+ .forEach(cte -> SessionState.getConsole().printInfo("CTE Suggestion:\n" + cte, false));
+ return result;
+ }
+
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CommonTableExpressionRegistry.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CommonTableExpressionRegistry.java
new file mode 100644
index 00000000000..6c130e7dc79
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CommonTableExpressionRegistry.java
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite;
+
+import org.apache.calcite.rel.RelNode;
+import org.apache.hadoop.hive.common.classification.InterfaceAudience;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.stream.Stream;
+
+/**
+ * A registry of common table expressions for a given query.
+ * <p>The registry is only meant to hold common expressions for a single query.</p>
+ * <p>The class is not thread-safe.</p>
+ */
+@InterfaceAudience.Private
+public final class CommonTableExpressionRegistry {
+ /**
+ * A unique collection of common table expressions.
+ * <p>The expressions may contain internal planning concepts such as {@link org.apache.calcite.plan.hep.HepRelVertex}.
+ * </p>
+ */
+ private final Map<String, RelNode> ctes = new HashMap<>();
+
+ /**
+ * Adds the specified common table expression to this registry.
+ * @param cte common table expression to be added to the registry.
+ */
+ public void add(RelNode cte) {
+ this.ctes.put(cte.getDigest(), cte);
+ }
+
+ /**
+ * @return a stream with all common table expression entries
+ */
+ public Stream<RelNode> entries() {
+ return ctes.values().stream().map(HiveCalciteUtil::stripHepVertices);
+ }
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CommonTableExpressionSuggester.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CommonTableExpressionSuggester.java
new file mode 100644
index 00000000000..694b9109998
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CommonTableExpressionSuggester.java
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite;
+
+import org.apache.calcite.rel.RelNode;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hive.common.classification.InterfaceAudience;
+import org.apache.hadoop.hive.common.classification.InterfaceStability;
+
+import java.util.List;
+
+/**
+ * Suggester for finding and returning interesting expressions that appear more than once in a query. The notion of
+ * "interesting" is specific to the actual implementation of this interface.
+ * <p>In some cases the interesting expressions may be readily available in the input query while in others they could be
+ * revealed after applying various transformations and algebraic equivalences.</p>
+ * <p>The final decision about using (or not) the suggestions provided by this class lies to the query optimizer. For
+ * various reasons (e.g, incorrect or expensive expressions) the optimizer may choose to reject/ignore the results
+ * provided by this class.</p>
+ * <p>Implementations of this interface must have a public no argument constructor since they are instantiated via reflection.
+ * The class is instantiated once for each query under compilation and then thrown away.</p>
+ * <p>The interface is experimental and subject to change without notice.</p>
+ */
+@InterfaceAudience.Public
+@InterfaceStability.Unstable
+public interface CommonTableExpressionSuggester {
+ /**
+ * Suggests interesting expressions for the specified query and configuration.
+ * @param input a relational expression representing the query under compilation.
+ * @param configuration the configuration to use for identifying and returning interesting expressions.
+ * @return a list with interesting expressions for the specified query
+ */
+ List<RelNode> suggest(RelNode input, Configuration configuration);
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CommonTableExpressionSuggesterFactory.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CommonTableExpressionSuggesterFactory.java
new file mode 100644
index 00000000000..249b85387d5
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CommonTableExpressionSuggesterFactory.java
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite;
+
+import org.apache.hadoop.hive.common.classification.InterfaceAudience;
+import org.apache.hadoop.hive.conf.HiveConf;
+
+import java.lang.reflect.InvocationTargetException;
+import java.util.Collections;
+
+@InterfaceAudience.Private
+public final class CommonTableExpressionSuggesterFactory {
+
+ private CommonTableExpressionSuggesterFactory() {
+ throw new IllegalStateException("Must not instantiate");
+ }
+
+ public static CommonTableExpressionSuggester create(HiveConf configuration) {
+ String suggestName = configuration.getVar(HiveConf.ConfVars.HIVE_CTE_SUGGESTER_CLASS);
+ if (suggestName == null || suggestName.isEmpty()) {
+ return (query, conf) -> Collections.emptyList();
+ }
+ try {
+ Class<?> suggesterClass = Class.forName(suggestName);
+ if (CommonTableExpressionSuggester.class.isAssignableFrom(suggesterClass)) {
+ return (CommonTableExpressionSuggester) suggesterClass.getDeclaredConstructor().newInstance();
+ }
+ throw new IllegalArgumentException(
+ suggesterClass.getSimpleName() + " must implement " + CommonTableExpressionSuggester.class.getSimpleName());
+ } catch (ClassNotFoundException | InstantiationException | IllegalAccessException | NoSuchMethodException | InvocationTargetException e) {
+ throw new CalciteCteException("Failed to instantiate suggester from class: " + suggestName, e);
+ }
+ }
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java
index fc48a623864..98f45476cd0 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java
@@ -31,6 +31,7 @@ import org.apache.calcite.plan.RelOptUtil;
import org.apache.calcite.plan.RelTraitSet;
import org.apache.calcite.plan.RelOptUtil.InputFinder;
import org.apache.calcite.plan.RelOptUtil.InputReferencedVisitor;
+import org.apache.calcite.plan.hep.HepRelVertex;
import org.apache.calcite.rel.RelNode;
import org.apache.calcite.rel.core.Aggregate;
import org.apache.calcite.rel.core.AggregateCall;
@@ -1351,4 +1352,20 @@ public class HiveCalciteUtil {
rexNode.accept(visitor);
return rexTableInputRefs;
}
+
+ public static RelNode stripHepVertices(RelNode rel) {
+ if (rel instanceof HepRelVertex) {
+ rel = ((HepRelVertex) rel).getCurrentRel();
+ }
+ List<RelNode> oldInputs = rel.getInputs();
+ List<RelNode> newInputs = new ArrayList<>();
+ for (RelNode oldInput : oldInputs) {
+ newInputs.add(stripHepVertices(oldInput));
+ }
+ if (oldInputs.equals(newInputs)) {
+ return rel;
+ } else {
+ return rel.copy(rel.getTraitSet(), newInputs);
+ }
+ }
}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java
index 7e7c1489cc1..1803e970633 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java
@@ -27,6 +27,7 @@ import org.apache.hadoop.hive.conf.HiveConf;
import org.apache.hadoop.hive.ql.optimizer.calcite.cost.HiveDefaultCostModel;
import org.apache.hadoop.hive.ql.optimizer.calcite.cost.HiveOnTezCostModel;
import org.apache.hadoop.hive.ql.optimizer.calcite.cost.HiveRelMdCost;
+import org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdAggregatedColumns;
import org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdColumnUniqueness;
import org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdCollation;
import org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdCumulativeCost;
@@ -40,6 +41,7 @@ import org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdRowCount;
import org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdRuntimeRowCount;
import org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdSelectivity;
import org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdSize;
+import org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdTableReferences;
import org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdUniqueKeys;
import com.google.common.collect.ImmutableList;
@@ -67,6 +69,8 @@ public class HiveDefaultRelMetadataProvider {
HiveRelMdDistribution.SOURCE,
HiveRelMdCollation.SOURCE,
HiveRelMdPredicates.SOURCE,
+ HiveRelMdTableReferences.SOURCE,
+ HiveRelMdAggregatedColumns.SOURCE,
JaninoRelMetadataProvider.DEFAULT)));
private final RelMetadataProvider metadataProvider;
@@ -102,6 +106,8 @@ public class HiveDefaultRelMetadataProvider {
HiveRelMdDistribution.SOURCE,
HiveRelMdCollation.SOURCE,
HiveRelMdPredicates.SOURCE,
+ HiveRelMdTableReferences.SOURCE,
+ HiveRelMdAggregatedColumns.SOURCE,
JaninoRelMetadataProvider.DEFAULT)));
if (nodeClasses != null) {
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelFactories.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelFactories.java
index 40fb932643c..ac2cef60fdd 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelFactories.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelFactories.java
@@ -49,6 +49,7 @@ import org.apache.calcite.tools.RelBuilderFactory;
import org.apache.calcite.util.ImmutableBitSet;
import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveAggregate;
import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveAntiJoin;
+import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableSpool;
import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveValues;
import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveExcept;
import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveFilter;
@@ -91,6 +92,8 @@ public class HiveRelFactories {
public static final SetOpFactory HIVE_SET_OP_FACTORY =
new HiveSetOpFactoryImpl();
+ public static final RelFactories.SpoolFactory HIVE_SPOOL_FACTORY = HiveTableSpool::new;
+
public static final RelBuilderFactory HIVE_BUILDER =
HiveRelBuilder.proto(
Contexts.of(
@@ -102,7 +105,8 @@ public class HiveRelFactories {
HIVE_SORT_EXCHANGE_FACTORY,
HIVE_VALUES_FACTORY,
HIVE_AGGREGATE_FACTORY,
- HIVE_SET_OP_FACTORY));
+ HIVE_SET_OP_FACTORY,
+ HIVE_SPOOL_FACTORY));
private HiveRelFactories() {
}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveSqlTypeUtil.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveSqlTypeUtil.java
new file mode 100644
index 00000000000..ea3e9704191
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveSqlTypeUtil.java
@@ -0,0 +1,49 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite;
+
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rel.type.RelDataTypeField;
+import org.apache.calcite.sql.type.SqlTypeName;
+
+import java.util.Objects;
+
+public final class HiveSqlTypeUtil {
+
+ private HiveSqlTypeUtil() {
+ throw new IllegalStateException("Must not instantiate.");
+ }
+ /**
+ * Returns whether the type contains the specified type name. If the type is a struct, map, or collection then
+ * containment is applied recursively on the respective parts (fields, component, key, value).
+ */
+ public static boolean containsSqlType(RelDataType type, SqlTypeName typeName) {
+ Objects.requireNonNull(typeName);
+ if (type == null) {
+ return false;
+ }
+ if (typeName.equals(type.getSqlTypeName())) {
+ return true;
+ }
+ if (type.isStruct()) {
+ return type.getFieldList().stream().map(RelDataTypeField::getType)
+ .anyMatch(t -> HiveSqlTypeUtil.containsSqlType(t, typeName));
+ }
+ return containsSqlType(type.getComponentType(), typeName) || containsSqlType(type.getKeyType(), typeName)
+ || containsSqlType(type.getValueType(), typeName);
+ }
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java
index 473c5d5fd5b..1fdc68108bc 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java
@@ -425,6 +425,10 @@ public class RelOptHiveTable implements RelOptTable {
columnPositions.build());
}
+ public void setRowCount(double rowCount){
+ this.rowCount = rowCount;
+ }
+
@Override
public double getRowCount() {
if (rowCount == -1) {
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableSpool.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableSpool.java
new file mode 100644
index 00000000000..d5a39835b87
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableSpool.java
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite.reloperators;
+
+import org.apache.calcite.plan.RelOptTable;
+import org.apache.calcite.plan.RelTraitSet;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.RelWriter;
+import org.apache.calcite.rel.core.Spool;
+import org.apache.calcite.rel.core.TableSpool;
+import org.apache.hadoop.hive.ql.optimizer.calcite.TraitsUtil;
+
+public class HiveTableSpool extends TableSpool implements HiveRelNode {
+ public HiveTableSpool(RelNode input, Type readType, Type writeType, RelOptTable table) {
+ super(input.getCluster(), TraitsUtil.getDefaultTraitSet(input.getCluster()), input, readType, writeType, table);
+ }
+
+ @Override
+ protected Spool copy(RelTraitSet traitSet, RelNode input, Type readType, Type writeType) {
+ return new HiveTableSpool(input, readType, writeType, table);
+ }
+
+ @Override
+ public RelWriter explainTerms(final RelWriter pw) {
+ return pw.input("input", getInput()).item("table", getTable().getQualifiedName());
+ }
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/CommonRelSubExprRegisterRule.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/CommonRelSubExprRegisterRule.java
new file mode 100644
index 00000000000..40b29f9a4ac
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/CommonRelSubExprRegisterRule.java
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite.rules;
+
+import org.apache.calcite.plan.CommonRelSubExprRule;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Aggregate;
+import org.apache.calcite.rel.core.Filter;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rel.core.Project;
+import org.apache.calcite.rel.metadata.RelMetadataQuery;
+import org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionRegistry;
+
+import com.google.common.collect.Multimap;
+
+import java.util.function.Predicate;
+
+import static org.apache.calcite.plan.RelRule.Config.EMPTY;
+
+/**
+ * Rule for saving relational expressions that appear more than once in a query tree to the planner context.
+ */
+public final class CommonRelSubExprRegisterRule extends CommonRelSubExprRule {
+ public static final CommonRelSubExprRegisterRule JOIN = new CommonRelSubExprRegisterRule(
+ o -> o.operand(Join.class).predicate(j -> JoinRelType.INNER == j.getJoinType()).anyInputs());
+ public static final CommonRelSubExprRegisterRule AGGREGATE =
+ new CommonRelSubExprRegisterRule(o -> o.operand(Aggregate.class).anyInputs());
+ public static final CommonRelSubExprRegisterRule FILTER =
+ new CommonRelSubExprRegisterRule(o -> o.operand(Filter.class).anyInputs());
+ public static final CommonRelSubExprRegisterRule PROJECT = new CommonRelSubExprRegisterRule(
+ o -> o.operand(Project.class).predicate(new InterestingRelNodePredicate()).anyInputs());
+
+ private CommonRelSubExprRegisterRule(OperandTransform operands) {
+ super(EMPTY.withOperandSupplier(operands).as(Config.class));
+ }
+
+ @Override
+ public void onMatch(final RelOptRuleCall call) {
+ CommonTableExpressionRegistry r = call.getPlanner().getContext().unwrap(CommonTableExpressionRegistry.class);
+ r.add(call.rel(0));
+ }
+
+ /**
+ * A predicate determining if a relational expression is interesting.
+ *
+ * The notion of interesting is loosely defined on purpose since it may change as the API evolves. At the moment an
+ * expression is considered interesting if it contains at least one of the following RelNode types:
+ * <ul>
+ * <li>{@link Join}</li>
+ * <li>{@link Aggregate}</li>
+ * <li>{@link Filter}</li>
+ * </ul>
+ */
+ private static final class InterestingRelNodePredicate implements Predicate<RelNode> {
+ @Override
+ public boolean test(final RelNode rel) {
+ RelMetadataQuery mq = rel.getCluster().getMetadataQuery();
+ Multimap<Class<? extends RelNode>, RelNode> types = mq.getNodeTypes(rel);
+ if (types == null) {
+ return false;
+ }
+ return types.keySet().stream().anyMatch(
+ t -> Join.class.isAssignableFrom(t) || Aggregate.class.isAssignableFrom(t) || Filter.class.isAssignableFrom(
+ t));
+ }
+ }
+
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/CteRuleConfig.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/CteRuleConfig.java
new file mode 100644
index 00000000000..a33b26303ac
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/CteRuleConfig.java
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite.rules;
+
+import org.apache.calcite.plan.RelOptRule;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.core.TableScan;
+import org.apache.calcite.util.ImmutableBeans;
+
+import java.util.List;
+import java.util.Map;
+
+public interface CteRuleConfig extends RelRule.Config {
+ CteRuleConfig DEFAULT = EMPTY.withOperandSupplier(b -> b.operand(TableScan.class).noInputs()).as(CteRuleConfig.class);
+
+ CteRuleConfig withTableOccurrences(Map<List<String>, Integer> tableOccurrences);
+ @ImmutableBeans.Property
+ Map<List<String>, Integer> getTableOccurrences();
+ CteRuleConfig withReferenceThreshold(int referenceThreshold);
+ @ImmutableBeans.Property
+ int referenceThreshold();
+
+ @Override
+ default RelOptRule toRule() {
+ throw new IllegalStateException("Must not be called");
+ }
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HivePartitionPruneRule.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HivePartitionPruneRule.java
index 28f097aa49a..cf0eefe72b9 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HivePartitionPruneRule.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HivePartitionPruneRule.java
@@ -50,6 +50,9 @@ public class HivePartitionPruneRule extends RelOptRule {
HiveTableScan tScan) {
// Original table
RelOptHiveTable hiveTable = (RelOptHiveTable) tScan.getTable();
+ if (hiveTable.getHiveTableMD().isMaterializedTable()) {
+ return;
+ }
// Copy original table scan and table
HiveTableScan tScanCopy = tScan.copyIncludingTable(tScan.getRowType());
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/RemoveInfrequentCteRule.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/RemoveInfrequentCteRule.java
new file mode 100644
index 00000000000..cbfb4133c10
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/RemoveInfrequentCteRule.java
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite.rules;
+
+import org.apache.calcite.plan.RelOptMaterialization;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.core.TableScan;
+
+/**
+ * Rule that removes CTEs from the plan by expanding the respective TableScans.
+ * <p>The rule assumes that all materializations registered in the planner refer to CTEs.</p>
+ */
+public class RemoveInfrequentCteRule extends RelRule<CteRuleConfig> {
+
+ public RemoveInfrequentCteRule(CteRuleConfig config) {
+ super(config);
+ if (config.referenceThreshold() <= 0) {
+ throw new IllegalArgumentException("Invalid reference threshold:" + config.referenceThreshold());
+ }
+ }
+
+ @Override
+ public boolean matches(final RelOptRuleCall call) {
+ TableScan scan = call.rel(0);
+ return config.getTableOccurrences().getOrDefault(scan.getTable().getQualifiedName(), 0)
+ <= config.referenceThreshold();
+ }
+
+ @Override
+ public void onMatch(RelOptRuleCall call) {
+ TableScan scan = call.rel(0);
+ String tableName = scan.getTable().getQualifiedName().toString();
+ for (RelOptMaterialization cte : call.getPlanner().getMaterializations()) {
+ if (tableName.equals(cte.qualifiedTableName.toString())) {
+ call.transformTo(cte.queryRel);
+ }
+ }
+ }
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/TableScanToSpoolRule.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/TableScanToSpoolRule.java
new file mode 100644
index 00000000000..19b7bf297e2
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/TableScanToSpoolRule.java
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite.rules;
+
+import org.apache.calcite.plan.RelOptMaterialization;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.prepare.RelOptTableImpl;
+import org.apache.calcite.rel.core.Spool;
+import org.apache.calcite.rel.core.TableScan;
+
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+import static org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories.HIVE_SPOOL_FACTORY;
+
+public class TableScanToSpoolRule extends RelRule<CteRuleConfig> {
+ /**
+ * Track created spools to avoid introducing more than one.
+ */
+ private final Set<String> spools = new HashSet<>();
+ public TableScanToSpoolRule(CteRuleConfig config) {
+ super(config);
+ if (config.referenceThreshold() <= 0) {
+ throw new IllegalArgumentException("Invalid reference threshold:" + config.referenceThreshold());
+ }
+ }
+
+ @Override
+ public boolean matches(final RelOptRuleCall call) {
+ TableScan scan = call.rel(0);
+ List<String> tableName = scan.getTable().getQualifiedName();
+ if (spools.contains(tableName.toString())) {
+ return false;
+ }
+ return config.getTableOccurrences().getOrDefault(tableName, 0) > config.referenceThreshold();
+ }
+
+ @Override
+ public void onMatch(RelOptRuleCall call) {
+ TableScan scan = call.rel(0);
+ String tableName = scan.getTable().getQualifiedName().toString();
+ for (RelOptMaterialization cte : call.getPlanner().getMaterializations()) {
+ if (tableName.equals(cte.qualifiedTableName.toString()) && spools.add(tableName)) {
+ RelOptTableImpl cteTable =
+ RelOptTableImpl.create(null, scan.getRowType(), scan.getTable().getQualifiedName(), null);
+ // The Spool types are not used at the moment so choice between LAZY/EAGER does not affect anything
+ call.transformTo(HIVE_SPOOL_FACTORY.createTableSpool(cte.queryRel, Spool.Type.LAZY, Spool.Type.LAZY, cteTable));
+ }
+ }
+ }
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/views/HiveMaterializedViewUtils.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/views/HiveMaterializedViewUtils.java
index fbd895dab6d..de3df0ecf2a 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/views/HiveMaterializedViewUtils.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/views/HiveMaterializedViewUtils.java
@@ -21,12 +21,14 @@ import com.google.common.base.Preconditions;
import com.google.common.collect.ImmutableList;
import java.math.BigDecimal;
import java.util.ArrayList;
+import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.Supplier;
import java.util.stream.Collectors;
@@ -46,6 +48,7 @@ import org.apache.calcite.rel.core.AggregateCall;
import org.apache.calcite.rel.core.Filter;
import org.apache.calcite.rel.core.Project;
import org.apache.calcite.rel.core.TableScan;
+import org.apache.calcite.rel.type.RelDataTypeField;
import org.apache.calcite.rex.RexBuilder;
import org.apache.calcite.rex.RexInputRef;
import org.apache.calcite.rex.RexNode;
@@ -56,8 +59,11 @@ import org.apache.hadoop.hive.common.TableName;
import org.apache.hadoop.hive.common.ValidTxnWriteIdList;
import org.apache.hadoop.hive.common.ValidWriteIdList;
import org.apache.hadoop.hive.common.type.SnapshotContext;
+import org.apache.hadoop.hive.conf.HiveConf;
import org.apache.hadoop.hive.metastore.api.FieldSchema;
+import org.apache.hadoop.hive.metastore.api.MetaException;
import org.apache.hadoop.hive.ql.ddl.DDLOperationContext;
+import org.apache.hadoop.hive.ql.exec.ColumnInfo;
import org.apache.hadoop.hive.ql.lockmgr.HiveTxnManager;
import org.apache.hadoop.hive.ql.lockmgr.LockException;
import org.apache.hadoop.hive.ql.metadata.Hive;
@@ -67,6 +73,7 @@ import org.apache.hadoop.hive.common.MaterializationSnapshot;
import org.apache.hadoop.hive.ql.metadata.HiveStorageHandler;
import org.apache.hadoop.hive.ql.metadata.MaterializedViewMetadata;
import org.apache.hadoop.hive.ql.metadata.Table;
+import org.apache.hadoop.hive.ql.optimizer.calcite.CalciteCteException;
import org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories;
import org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable;
import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveFilter;
@@ -74,12 +81,15 @@ import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveGroupingID;
import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveProject;
import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveRelNode;
import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableScan;
+import org.apache.hadoop.hive.ql.optimizer.calcite.translator.TypeConverter;
import org.apache.hadoop.hive.ql.parse.DruidSqlOperatorConverter;
+import org.apache.hadoop.hive.ql.parse.QueryTables;
import org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAccessControlException;
import org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzContext;
import org.apache.hadoop.hive.ql.security.authorization.plugin.HiveOperationType;
import org.apache.hadoop.hive.ql.security.authorization.plugin.HivePrivilegeObject;
import org.apache.hadoop.hive.ql.session.SessionState;
+import org.apache.hadoop.hive.serde2.typeinfo.TypeInfo;
import org.apache.hive.common.util.TxnIdUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -536,4 +546,33 @@ public class HiveMaterializedViewUtils {
}
return snapshot;
}
+
+ public static RelOptMaterialization createCTEMaterialization(String viewName, RelNode body, HiveConf conf) {
+ RelOptCluster cluster = body.getCluster();
+ List<ColumnInfo> columns = new ArrayList<>();
+ for (RelDataTypeField f : body.getRowType().getFieldList()) {
+ TypeInfo info = TypeConverter.convert(f.getType());
+ columns.add(new ColumnInfo(f.getName(), info, f.getType().isNullable(), viewName, false, false));
+ }
+ List<String> fullName = Arrays.asList("cte", viewName);
+ org.apache.hadoop.hive.metastore.api.Table metaTable = Table.getEmptyTable("cte", viewName);
+ metaTable.setTemporary(true);
+ try {
+ // Setting a location avoids a NPE when fetching statistics
+ metaTable.getSd().setLocation(SessionState.generateTempTableLocation(conf));
+ } catch (MetaException e) {
+ throw new CalciteCteException("Failed to create temporary location", e);
+ }
+ Table hiveTable = new Table(metaTable);
+ hiveTable.setMaterializedTable(true);
+ RelOptHiveTable optTable =
+ new RelOptHiveTable(null, cluster.getTypeFactory(), fullName, body.getRowType(), hiveTable, columns,
+ Collections.emptyList(), Collections.emptyList(), new HiveConf(), new QueryTables(true), new HashMap<>(),
+ new HashMap<>(), new AtomicInteger());
+ optTable.setRowCount(cluster.getMetadataQuery().getRowCount(body));
+ final TableScan scan =
+ new HiveTableScan(cluster, cluster.traitSetOf(HiveRelNode.CONVENTION), optTable, viewName, null, false, false);
+
+ return new RelOptMaterialization(scan, body, null, fullName);
+ }
}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/AggregatedColumns.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/AggregatedColumns.java
new file mode 100644
index 00000000000..c65a733e014
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/AggregatedColumns.java
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite.stats;
+
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.metadata.Metadata;
+import org.apache.calcite.rel.metadata.MetadataDef;
+import org.apache.calcite.rel.metadata.MetadataHandler;
+import org.apache.calcite.rel.metadata.RelMetadataQuery;
+import org.apache.calcite.util.ImmutableBitSet;
+
+import javax.annotation.Nullable;
+
+/**
+ * Metadata about whether a set of columns originates from aggregation functions.
+ */
+public interface AggregatedColumns extends Metadata {
+
+ MetadataDef<AggregatedColumns> DEF = MetadataDef.of(AggregatedColumns.class, AggregatedColumns.Handler.class,
+ MetadataMethod.AGGREGATED_COLUMNS.method());
+
+ /**
+ * Determines whether the specified set of columns originates from aggregation functions.
+ *
+ * @param columns column mask representing a subset of columns for the current relational expression.
+ * @return whether the specified columns originate from aggregate functions or null if there is not enough information
+ * to infer the origin function.
+ */
+ @Nullable
+ Boolean areColumnsAggregated(final ImmutableBitSet columns);
+
+ interface Handler extends MetadataHandler<AggregatedColumns> {
+ @Nullable
+ Boolean areColumnsAggregated(RelNode r, RelMetadataQuery mq, final ImmutableBitSet columns);
+ }
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdAggregatedColumns.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdAggregatedColumns.java
new file mode 100644
index 00000000000..daaa1493154
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdAggregatedColumns.java
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite.stats;
+
+import org.apache.calcite.plan.hep.HepRelVertex;
+import org.apache.calcite.plan.volcano.RelSubset;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Aggregate;
+import org.apache.calcite.rel.core.Filter;
+import org.apache.calcite.rel.core.Project;
+import org.apache.calcite.rel.metadata.MetadataDef;
+import org.apache.calcite.rel.metadata.MetadataHandler;
+import org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider;
+import org.apache.calcite.rel.metadata.RelMetadataProvider;
+import org.apache.calcite.rel.metadata.RelMetadataQuery;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.util.ImmutableBitSet;
+
+import javax.annotation.Nullable;
+
+public class HiveRelMdAggregatedColumns implements MetadataHandler<AggregatedColumns> {
+ public static final RelMetadataProvider SOURCE =
+ ReflectiveRelMetadataProvider.reflectiveSource(new HiveRelMdAggregatedColumns(),
+ MetadataMethod.AGGREGATED_COLUMNS.method());
+ @Nullable
+ public Boolean areColumnsAggregated(Project rel, RelMetadataQuery mq, ImmutableBitSet columns) {
+ ImmutableBitSet.Builder iRefs = ImmutableBitSet.builder();
+ for (int bit : columns) {
+ RexNode e = rel.getProjects().get(bit);
+ if (e instanceof RexInputRef) {
+ iRefs.set(((RexInputRef) e).getIndex());
+ } else {
+ return false;
+ }
+ }
+ return ((HiveRelMetadataQuery) mq).areColumnsAggregated(rel.getInput(), iRefs.build());
+ }
+ @Nullable
+ public Boolean areColumnsAggregated(Aggregate aggregate, RelMetadataQuery mq, ImmutableBitSet columns) {
+ ImmutableBitSet aggColumns =
+ ImmutableBitSet.range(aggregate.getGroupCount(), aggregate.getGroupCount() + aggregate.getAggCallList().size());
+ return aggColumns.equals(columns);
+ }
+ @Nullable
+ public Boolean areColumnsAggregated(Filter rel, RelMetadataQuery mq, ImmutableBitSet columns) {
+ return ((HiveRelMetadataQuery) mq).areColumnsAggregated(rel.getInput(), columns);
+ }
+ @Nullable
+ public Boolean areColumnsAggregated(HepRelVertex rel, RelMetadataQuery mq, ImmutableBitSet columns) {
+ return ((HiveRelMetadataQuery) mq).areColumnsAggregated(rel.getCurrentRel(), columns);
+ }
+ @Nullable
+ public Boolean areColumnsAggregated(RelSubset rel, RelMetadataQuery mq, ImmutableBitSet columns) {
+ return ((HiveRelMetadataQuery) mq).areColumnsAggregated(rel.getOriginal(), columns);
+ }
+ @Nullable
+ public Boolean areColumnsAggregated(RelNode rel, RelMetadataQuery mq, ImmutableBitSet columns) {
+ return null;
+ }
+
+ @Override
+ public MetadataDef<AggregatedColumns> getDef() {
+ return AggregatedColumns.DEF;
+ }
+
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdColumnUniqueness.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdColumnUniqueness.java
index 2dede97723b..3fff3a8cc01 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdColumnUniqueness.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdColumnUniqueness.java
@@ -18,6 +18,7 @@
package org.apache.hadoop.hive.ql.optimizer.calcite.stats;
+import org.apache.calcite.rel.core.Spool;
import org.apache.calcite.rel.metadata.BuiltInMetadata;
import org.apache.calcite.rel.metadata.MetadataDef;
import org.apache.calcite.rel.metadata.MetadataHandler;
@@ -57,6 +58,10 @@ public final class HiveRelMdColumnUniqueness
return tbl.isNonNullableKey(columns);
}
}
+
+ public Boolean areColumnsUnique(Spool rel, RelMetadataQuery mq, ImmutableBitSet columns, boolean ignoreNulls) {
+ return mq.areColumnsUnique(rel.getInput(), columns, ignoreNulls);
+ }
}
// End HiveRelMdColumnUniqueness.java
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistinctRowCount.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistinctRowCount.java
index d550a11698c..b0f40d0d815 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistinctRowCount.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistinctRowCount.java
@@ -21,6 +21,7 @@ import org.apache.calcite.plan.RelOptUtil;
import org.apache.calcite.rel.RelNode;
import org.apache.calcite.rel.convert.Converter;
import org.apache.calcite.rel.core.JoinRelType;
+import org.apache.calcite.rel.core.Spool;
import org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider;
import org.apache.calcite.rel.metadata.RelMdDistinctRowCount;
import org.apache.calcite.rel.metadata.RelMdUtil;
@@ -65,6 +66,11 @@ public class HiveRelMdDistinctRowCount extends RelMdDistinctRowCount {
return Math.min(noDistinctRows, mq.getRowCount(htRel));
}
+ public Double getDistinctRowCount(Spool spool, RelMetadataQuery mq, ImmutableBitSet groupKey,
+ RexNode predicate) {
+ return mq.getDistinctRowCount(spool.getInput(), groupKey, predicate);
+ }
+
public static Double getDistinctRowCount(RelNode r, RelMetadataQuery mq, int indx) {
ImmutableBitSet bitSetOfRqdProj = ImmutableBitSet.of(indx);
return mq.getDistinctRowCount(r, bitSetOfRqdProj, r
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdTableReferences.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdTableReferences.java
new file mode 100644
index 00000000000..d141a2b8431
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdTableReferences.java
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite.stats;
+
+import org.apache.calcite.rel.SingleRel;
+import org.apache.calcite.rel.metadata.BuiltInMetadata;
+import org.apache.calcite.rel.metadata.MetadataDef;
+import org.apache.calcite.rel.metadata.MetadataHandler;
+import org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider;
+import org.apache.calcite.rel.metadata.RelMetadataProvider;
+import org.apache.calcite.rel.metadata.RelMetadataQuery;
+import org.apache.calcite.rex.RexTableInputRef;
+import org.apache.calcite.util.BuiltInMethod;
+
+import java.util.Set;
+
+public class HiveRelMdTableReferences implements MetadataHandler<BuiltInMetadata.TableReferences> {
+
+ public static final RelMetadataProvider SOURCE =
+ ReflectiveRelMetadataProvider.reflectiveSource(new HiveRelMdTableReferences(),
+ BuiltInMethod.TABLE_REFERENCES.method);
+
+ @Override
+ public MetadataDef<BuiltInMetadata.TableReferences> getDef() {
+ return BuiltInMetadata.TableReferences.DEF;
+ }
+
+ public Set<RexTableInputRef.RelTableRef> getTableReferences(SingleRel r, RelMetadataQuery mq) {
+ return mq.getTableReferences(r.getInput());
+ }
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMetadataQuery.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMetadataQuery.java
new file mode 100644
index 00000000000..8c7d8373a70
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMetadataQuery.java
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite.stats;
+
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.metadata.JaninoRelMetadataProvider;
+import org.apache.calcite.rel.metadata.RelMetadataQuery;
+import org.apache.calcite.util.ImmutableBitSet;
+
+import javax.annotation.Nullable;
+
+public class HiveRelMetadataQuery extends RelMetadataQuery {
+ private AggregatedColumns.Handler aggOriginHandler;
+
+ public HiveRelMetadataQuery() {
+ super();
+ this.aggOriginHandler = initialHandler(AggregatedColumns.Handler.class);
+ }
+
+ /**
+ * Returns the {@link AggregatedColumns} statistic.
+ *
+ * @param input the input relational expression
+ * @param columns mask representing a subset of columns for the input relational expression.
+ * @return whether the specified columns originate from aggregate functions or null if there is not enough information
+ * to infer the origin function.
+ */
+ @Nullable
+ public Boolean areColumnsAggregated(RelNode input, ImmutableBitSet columns) {
+ for (;;) {
+ try {
+ return aggOriginHandler.areColumnsAggregated(input, this, columns);
+ } catch (JaninoRelMetadataProvider.NoHandler e) {
+ aggOriginHandler = revise(e.relClass, AggregatedColumns.DEF);
+ }
+ }
+ }
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/MetadataMethod.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/MetadataMethod.java
new file mode 100644
index 00000000000..8127ef6ad79
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/MetadataMethod.java
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite.stats;
+
+import org.apache.calcite.linq4j.tree.Types;
+import org.apache.calcite.rel.metadata.Metadata;
+import org.apache.calcite.util.ImmutableBitSet;
+
+import java.lang.reflect.Method;
+
+public enum MetadataMethod {
+ AGGREGATED_COLUMNS(AggregatedColumns.class, "areColumnsAggregated", ImmutableBitSet.class);
+ private final Method method;
+
+ MetadataMethod(Class<? extends Metadata> metaClass, String name, Class<?> argClass) {
+ this.method = Types.lookupMethod(metaClass, name, argClass);
+ }
+
+ public Method method() {
+ return method;
+ }
+}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTBuilder.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTBuilder.java
index f15dbf5694d..12a0bcaffe0 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTBuilder.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTBuilder.java
@@ -77,6 +77,9 @@ public class ASTBuilder {
assert hts != null;
RelOptHiveTable hTbl = (RelOptHiveTable) hts.getTable();
+ if (hTbl.getHiveTableMD().isMaterializedTable()) {
+ return cte(hTbl.getHiveTableMD().getTableName(), hts.getTableAlias());
+ }
ASTBuilder tableNameBuilder = ASTBuilder.construct(HiveParser.TOK_TABNAME, "TOK_TABNAME")
.add(HiveParser.Identifier, hTbl.getHiveTableMD().getDbName())
.add(HiveParser.Identifier, hTbl.getHiveTableMD().getTableName());
@@ -187,6 +190,15 @@ public class ASTBuilder {
return b.node();
}
+ /**
+ * @return an ASTNode representing a (table) reference to a CTE with the specified name and alias.
+ */
+ public static ASTNode cte(String name, final String alias) {
+ return ASTBuilder.construct(HiveParser.TOK_TABREF, "TOK_TABREF")
+ .add(ASTBuilder.construct(HiveParser.TOK_TABNAME, "TOK_TABNAME").add(HiveParser.Identifier, name))
+ .add(HiveParser.Identifier, alias).node();
+ }
+
public static ASTNode join(ASTNode left, ASTNode right, JoinRelType joinType, ASTNode cond) {
ASTBuilder b = null;
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTConverter.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTConverter.java
index 2c0fe365dac..8c6ff8bab0e 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTConverter.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTConverter.java
@@ -45,6 +45,7 @@ import org.apache.calcite.rel.core.Project;
import org.apache.calcite.rel.core.Sort;
import org.apache.calcite.rel.core.TableFunctionScan;
import org.apache.calcite.rel.core.TableScan;
+import org.apache.calcite.rel.core.TableSpool;
import org.apache.calcite.rel.core.Union;
import org.apache.calcite.rel.core.Values;
import org.apache.calcite.rel.metadata.RelMetadataQuery;
@@ -114,6 +115,7 @@ public class ASTConverter {
private Filter having;
private RelNode select;
private RelNode orderLimit;
+ private List<ASTNode> ctes;
private Schema schema;
@@ -121,18 +123,23 @@ public class ASTConverter {
private PlanMapper planMapper;
- ASTConverter(RelNode root, long dtCounterInitVal, PlanMapper planMapper) {
+ ASTConverter(RelNode root, long dtCounterInitVal, PlanMapper planMapper, List<ASTNode> ctes) {
this.root = root;
hiveAST = new HiveAST();
this.derivedTableCount = dtCounterInitVal;
this.planMapper = planMapper;
+ this.ctes = ctes;
}
public static ASTNode convert(final RelNode relNode, List<FieldSchema> resultSchema, boolean alignColumns, PlanMapper planMapper)
throws CalciteSemanticException {
RelNode root = PlanModifierForASTConv.convertOpTree(relNode, resultSchema, alignColumns);
- ASTConverter c = new ASTConverter(root, 0, planMapper);
- return c.convert();
+ ASTConverter c = new ASTConverter(root, 0, planMapper, new ArrayList<>());
+ ASTNode r = c.convert();
+ for (ASTNode cte : c.ctes) {
+ r.insertChild(0, cte);
+ }
+ return r;
}
/**
@@ -571,10 +578,10 @@ public class ASTConverter {
}
} else if (r instanceof Union) {
Union u = ((Union) r);
- ASTNode left = new ASTConverter(((Union) r).getInput(0), this.derivedTableCount, planMapper).convert();
+ ASTNode left = new ASTConverter(((Union) r).getInput(0), this.derivedTableCount, planMapper, this.ctes).convert();
for (int ind = 1; ind < u.getInputs().size(); ind++) {
left = getUnionAllAST(left, new ASTConverter(((Union) r).getInput(ind),
- this.derivedTableCount, planMapper).convert());
+ this.derivedTableCount, planMapper, this.ctes).convert());
String sqAlias = nextAlias();
ast = ASTBuilder.subQuery(left, sqAlias);
s = new Schema((Union) r, sqAlias);
@@ -590,8 +597,17 @@ public class ASTConverter {
ast = createASTLateralView(tfs, s, tableFunctionSource, sqAlias);
+ } else if (r instanceof TableSpool) {
+ TableSpool spool = (TableSpool) r;
+ ASTConverter cteConverter =
+ new ASTConverter(spool.getInput(), this.derivedTableCount, planMapper, Collections.emptyList());
+ String cteName = Iterables.getLast(spool.getTable().getQualifiedName());
+ ASTNode cte = ASTBuilder.createAST(HiveParser.TOK_CTE, "TOK_CTE");
+ cte.addChild(ASTBuilder.subQuery(cteConverter.convert(), cteName));
+ ctes.add(cte);
+ return new QueryBlockInfo(cteConverter.getRowSchema(cteName), ASTBuilder.cte(cteName, cteName));
} else {
- ASTConverter src = new ASTConverter(r, this.derivedTableCount, planMapper);
+ ASTConverter src = new ASTConverter(r, this.derivedTableCount, planMapper, this.ctes);
ASTNode srcAST = src.convert();
String sqAlias = nextAlias();
s = src.getRowSchema(sqAlias);
@@ -757,6 +773,8 @@ public class ASTConverter {
ASTConverter.this.from = node;
} else if (node instanceof Union) {
ASTConverter.this.from = node;
+ } else if (node instanceof TableSpool) {
+ ASTConverter.this.from = node;
} else if (node instanceof Aggregate) {
ASTConverter.this.groupBy = (Aggregate) node;
} else if (node instanceof Sort || node instanceof Exchange) {
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/PlanModifierForASTConv.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/PlanModifierForASTConv.java
index f2f41f18133..8ed19079dd1 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/PlanModifierForASTConv.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/PlanModifierForASTConv.java
@@ -39,6 +39,7 @@ import org.apache.calcite.rel.core.Join;
import org.apache.calcite.rel.core.Project;
import org.apache.calcite.rel.core.SetOp;
import org.apache.calcite.rel.core.Sort;
+import org.apache.calcite.rel.core.Spool;
import org.apache.calcite.rel.core.Window.RexWinAggCall;
import org.apache.calcite.rel.rules.MultiJoin;
import org.apache.calcite.rel.type.RelDataType;
@@ -46,6 +47,7 @@ import org.apache.calcite.rel.type.RelDataTypeFactory;
import org.apache.calcite.rex.RexNode;
import org.apache.calcite.rex.RexOver;
import org.apache.calcite.sql.SqlAggFunction;
+import org.apache.calcite.tools.RelBuilder;
import org.apache.calcite.util.Pair;
import org.apache.hadoop.hive.metastore.api.FieldSchema;
import org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException;
@@ -211,6 +213,12 @@ public class PlanModifierForASTConv {
// called recursively, it will have the correct parent.
rel = newParent.getInputs().get(0);
}
+ } else if (rel instanceof Spool) {
+ Spool spool = (Spool) rel;
+ RelBuilder b = HiveRelFactories.HIVE_BUILDER.create(spool.getCluster(),null);
+ b.push(spool.getInput());
+ b.project(b.fields(), spool.getTable().getRowType().getFieldNames(), true);
+ spool.replaceInput(0, b.build());
}
}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java b/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
index 72c195aaefe..f8bf8722e92 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
@@ -29,6 +29,7 @@ import com.google.common.collect.Multimap;
import java.util.Map.Entry;
import java.util.Optional;
+import java.util.function.Function;
import java.util.regex.Pattern;
import org.antlr.runtime.ClassicToken;
import org.antlr.runtime.CommonToken;
@@ -55,10 +56,12 @@ import org.apache.calcite.config.CalciteConnectionProperty;
import org.apache.calcite.config.NullCollation;
import org.apache.calcite.interpreter.BindableConvention;
import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptCostImpl;
import org.apache.calcite.plan.RelOptMaterialization;
import org.apache.calcite.plan.RelOptPlanner;
import org.apache.calcite.plan.RelOptRule;
import org.apache.calcite.plan.RelOptSchema;
+import org.apache.calcite.plan.RelOptTable;
import org.apache.calcite.plan.RelOptUtil;
import org.apache.calcite.plan.RelTraitSet;
import org.apache.calcite.plan.hep.HepMatchOrder;
@@ -162,16 +165,21 @@ import org.apache.hadoop.hive.ql.metadata.VirtualColumn;
import org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException;
import org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSubquerySemanticException;
import org.apache.hadoop.hive.ql.optimizer.calcite.CalciteViewSemanticException;
+import org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionSuggester;
+import org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionSuggesterFactory;
import org.apache.hadoop.hive.ql.optimizer.calcite.HiveCalciteUtil;
import org.apache.hadoop.hive.ql.optimizer.calcite.HiveConfPlannerContext;
import org.apache.hadoop.hive.ql.optimizer.calcite.HiveDefaultRelMetadataProvider;
import org.apache.hadoop.hive.ql.optimizer.calcite.HiveMaterializedViewASTSubQueryRewriteShuttle;
+import org.apache.hadoop.hive.ql.optimizer.calcite.HiveSqlTypeUtil;
import org.apache.hadoop.hive.ql.optimizer.calcite.HiveTezModelRelMetadataProvider;
import org.apache.hadoop.hive.ql.optimizer.calcite.RuleEventLogger;
+import org.apache.hadoop.hive.ql.optimizer.calcite.rules.CteRuleConfig;
import org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveAggregateSortLimitRule;
import org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveJoinSwapConstraintsRule;
import org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRemoveEmptySingleRules;
import org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveSemiJoinProjectTransposeRule;
+import org.apache.hadoop.hive.ql.optimizer.calcite.rules.RemoveInfrequentCteRule;
import org.apache.hadoop.hive.ql.optimizer.calcite.rules.jdbc.JDBCAggregateProjectMergeRule;
import org.apache.hadoop.hive.ql.optimizer.calcite.rules.views.HiveMaterializationRelMetadataProvider;
import org.apache.hadoop.hive.ql.optimizer.calcite.HivePlannerContext;
@@ -267,6 +275,7 @@ import org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveUnionMergeRule;
import org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveUnionPullUpConstantsRule;
import org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveWindowingFixRule;
import org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveWindowingLastValueRewrite;
+import org.apache.hadoop.hive.ql.optimizer.calcite.rules.TableScanToSpoolRule;
import org.apache.hadoop.hive.ql.optimizer.calcite.rules.jdbc.JDBCAbstractSplitFilterRule;
import org.apache.hadoop.hive.ql.optimizer.calcite.rules.jdbc.JDBCAggregationPushDownRule;
import org.apache.hadoop.hive.ql.optimizer.calcite.rules.jdbc.JDBCExpandExpressionsRule;
@@ -280,6 +289,7 @@ import org.apache.hadoop.hive.ql.optimizer.calcite.rules.jdbc.JDBCUnionPushDownR
import org.apache.hadoop.hive.ql.optimizer.calcite.rules.views.HiveMaterializedViewBoxing;
import org.apache.hadoop.hive.ql.optimizer.calcite.rules.views.HiveMaterializedViewRule;
import org.apache.hadoop.hive.ql.optimizer.calcite.rules.views.HiveMaterializedViewUtils;
+import org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMetadataQuery;
import org.apache.hadoop.hive.ql.optimizer.calcite.translator.ASTBuilder;
import org.apache.hadoop.hive.ql.optimizer.calcite.translator.ASTConverter;
import org.apache.hadoop.hive.ql.parse.type.FunctionHelper;
@@ -633,7 +643,7 @@ public class CalcitePlanner extends SemanticAnalyzer {
// unfortunately making prunedPartitions immutable is not possible
// here with SemiJoins not all tables are costed in CBO, so their
// PartitionList is not evaluated until the run phase.
- getMetaData(getQB());
+ getMetaData(getQB(), true);
disableJoinMerge = defaultJoinMerge;
sinkOp = genPlan(getQB());
@@ -1653,6 +1663,7 @@ public class CalcitePlanner extends SemanticAnalyzer {
// Create and set MD provider
HiveDefaultRelMetadataProvider mdProvider = new HiveDefaultRelMetadataProvider(conf, HIVE_REL_NODE_CLASSES);
RelMetadataQuery.THREAD_PROVIDERS.set(JaninoRelMetadataProvider.of(mdProvider.getMetadataProvider()));
+ optCluster.setMetadataQuerySupplier(HiveRelMetadataQuery::new);
optCluster.invalidateMetadataQuery();
calcitePlan = applyMaterializedViewRewritingByText(
@@ -1738,6 +1749,14 @@ public class CalcitePlanner extends SemanticAnalyzer {
perfLogger.perfLogBegin(this.getClass().getName(), PerfLogger.POSTJOIN_ORDERING);
calcitePlan = applyPostJoinOrderingTransform(calcitePlan, mdProvider.getMetadataProvider(), executorProvider);
perfLogger.perfLogEnd(this.getClass().getName(), PerfLogger.POSTJOIN_ORDERING);
+ // Perform the CTE rewriting near the end of CBO transformations to avoid interference of the new HiveTableSpool
+ // operator with other rules (especially those related to constant folding and branch pruning).
+ if (!forViewCreation) {
+ calcitePlan = applyCteRewriting(planner, calcitePlan, mdProvider.getMetadataProvider(), executorProvider);
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Plan after CTE rewriting:\n{}", RelOptUtil.toString(calcitePlan));
+ }
+ }
perfLogger.perfLogBegin(this.getClass().getName(), PerfLogger.HIVE_SORT_PREDICATES);
if (conf.getBoolVar(HiveConf.ConfVars.HIVE_OPTIMIZE_SORT_PREDS_WITH_STATS)) {
calcitePlan = calcitePlan.accept(new HiveFilterSortPredicates(noColsMissingStats));
@@ -1981,7 +2000,6 @@ public class CalcitePlanner extends SemanticAnalyzer {
final boolean useMaterializedViewsRegistry =
!conf.get(HiveConf.ConfVars.HIVE_SERVER2_MATERIALIZED_VIEWS_REGISTRY_IMPL.varname).equals("DUMMY");
- final String ruleExclusionRegex = conf.get(ConfVars.HIVE_CBO_RULE_EXCLUSION_REGEX.varname, "");
final RelNode calcitePreMVRewritingPlan = basePlan;
final Set<TableName> tablesUsedQuery = getTablesUsed(basePlan);
@@ -2016,6 +2034,49 @@ public class CalcitePlanner extends SemanticAnalyzer {
return calcitePreMVRewritingPlan;
}
+ basePlan = rewriteUsingViews(planner, basePlan, mdProvider, executorProvider, materializations);
+
+ List<Table> materializedViewsUsedOriginalPlan = getMaterializedViewsUsed(calcitePreMVRewritingPlan);
+ List<Table> materializedViewsUsedAfterRewrite = getMaterializedViewsUsed(basePlan);
+ if (materializedViewsUsedOriginalPlan.size() == materializedViewsUsedAfterRewrite.size()) {
+ // Materialized view-based rewriting did not happen, we can return the original plan
+ return calcitePreMVRewritingPlan;
+ }
+
+ try {
+ if (!HiveMaterializedViewUtils.checkPrivilegeForMaterializedViews(materializedViewsUsedAfterRewrite)) {
+ // if materialized views do not have appropriate privileges, we shouldn't be using them
+ return calcitePreMVRewritingPlan;
+ }
+ } catch (HiveException e) {
+ LOG.warn("Exception checking privileges for materialized views", e);
+ return calcitePreMVRewritingPlan;
+ }
+ // A rewriting was produced, we will check whether it was part of an incremental rebuild
+ // to try to replace INSERT OVERWRITE by INSERT or MERGE
+ if (useMaterializedViewsRegistry) {
+ // Before proceeding we need to check whether materialized views used are up-to-date
+ // wrt information in metastore
+ try {
+ if (!db.validateMaterializedViewsFromRegistry(materializedViewsUsedAfterRewrite, tablesUsedQuery,
+ queryState::getValidTxnList, getTxnMgr())) {
+ return calcitePreMVRewritingPlan;
+ }
+ } catch (HiveException e) {
+ LOG.warn("Exception validating materialized views", e);
+ return calcitePreMVRewritingPlan;
+ }
+ }
+ // Now we trigger some needed optimization rules again
+ return applyPreJoinOrderingTransforms(basePlan, mdProvider, executorProvider);
+ }
+
+ private RelNode rewriteUsingViews(RelOptPlanner planner, RelNode basePlan,
+ RelMetadataProvider mdProvider, RexExecutor executorProvider, Collection<? extends RelOptMaterialization> materializations) {
+ final PerfLogger perfLogger = SessionState.getPerfLogger();
+ perfLogger.perfLogBegin(this.getClass().getName(), PerfLogger.OPTIMIZER);
+ final RelOptCluster optCluster = basePlan.getCluster();
+
// We need to expand IN/BETWEEN expressions when materialized view rewriting
// is triggered since otherwise this may prevent some rewritings from happening
HepProgramBuilder program = new HepProgramBuilder();
@@ -2035,6 +2096,10 @@ public class CalcitePlanner extends SemanticAnalyzer {
// Add materializations to planner
for (RelOptMaterialization materialization : materializations) {
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("Adding materialization {} to the planner; the plan is:\n{}", materialization.qualifiedTableName,
+ RelOptUtil.toString(materialization.queryRel));
+ }
planner.addMaterialization(materialization);
}
// Add rule to split aggregate with grouping sets (if any)
@@ -2050,6 +2115,7 @@ public class CalcitePlanner extends SemanticAnalyzer {
planner.addRule(new HivePartitionPruneRule(conf));
// Optimize plan
+ final String ruleExclusionRegex = conf.get(ConfVars.HIVE_CBO_RULE_EXCLUSION_REGEX.varname, "");
if (!ruleExclusionRegex.isEmpty()) {
LOG.info("The CBO rules matching the following regex are excluded from planning: {}",
ruleExclusionRegex);
@@ -2064,39 +2130,64 @@ public class CalcitePlanner extends SemanticAnalyzer {
optCluster.invalidateMetadataQuery();
RelMetadataQuery.THREAD_PROVIDERS.set(JaninoRelMetadataProvider.of(mdProvider));
- List<Table> materializedViewsUsedOriginalPlan = getMaterializedViewsUsed(calcitePreMVRewritingPlan);
- List<Table> materializedViewsUsedAfterRewrite = getMaterializedViewsUsed(basePlan);
- if (materializedViewsUsedOriginalPlan.size() == materializedViewsUsedAfterRewrite.size()) {
- // Materialized view-based rewriting did not happen, we can return the original plan
- return calcitePreMVRewritingPlan;
- }
+ perfLogger.perfLogEnd(this.getClass().getName(), PerfLogger.OPTIMIZER, "Calcite: View-based rewriting");
+ return basePlan;
+ }
- try {
- if (!HiveMaterializedViewUtils.checkPrivilegeForMaterializedViews(materializedViewsUsedAfterRewrite)) {
- // if materialized views do not have appropriate privileges, we shouldn't be using them
- return calcitePreMVRewritingPlan;
- }
- } catch (HiveException e) {
- LOG.warn("Exception checking privileges for materialized views", e);
- return calcitePreMVRewritingPlan;
+ private RelNode applyCteRewriting(RelOptPlanner planner, RelNode basePlan, RelMetadataProvider mdProvider, RexExecutor executorProvider) {
+ final int referenceThreshold = conf.getIntVar(ConfVars.HIVE_CTE_MATERIALIZE_THRESHOLD);
+ if (referenceThreshold <= 0) {
+ return basePlan;
}
- // A rewriting was produced, we will check whether it was part of an incremental rebuild
- // to try to replace INSERT OVERWRITE by INSERT or MERGE
- if (useMaterializedViewsRegistry) {
- // Before proceeding we need to check whether materialized views used are up-to-date
- // wrt information in metastore
- try {
- if (!db.validateMaterializedViewsFromRegistry(materializedViewsUsedAfterRewrite, tablesUsedQuery,
- queryState::getValidTxnList, getTxnMgr())) {
- return calcitePreMVRewritingPlan;
- }
- } catch (HiveException e) {
- LOG.warn("Exception validating materialized views", e);
- return calcitePreMVRewritingPlan;
+ CommonTableExpressionSuggester suggester = CommonTableExpressionSuggesterFactory.create(conf);
+ List<RelNode> ctes = suggester.suggest(basePlan, conf);
+ if (ctes.isEmpty()) {
+ return basePlan;
+ }
+ HiveRelMetadataQuery mq = (HiveRelMetadataQuery) basePlan.getCluster().getMetadataQuery();
+ List<RelOptMaterialization> cteMVs = new ArrayList<>();
+ for (int i = 0; i < ctes.size(); i++) {
+ final RelNode cte = ctes.get(i);
+ if (isMaterializableCte(mq, cte)) {
+ cteMVs.add(HiveMaterializedViewUtils.createCTEMaterialization("cte_suggestion_" + i, cte, conf));
}
}
- // Now we trigger some needed optimization rules again
- return applyPreJoinOrderingTransforms(basePlan, mdProvider, executorProvider);
+ final RelNode ctePlan = rewriteUsingViews(planner, basePlan, mdProvider, executorProvider, cteMVs);
+ Map<List<String>, Integer> tableOccurrences = RelOptUtil.findAllTables(ctePlan).stream()
+ .map(RelOptTable::getQualifiedName)
+ .collect(Collectors.toMap(Function.identity(), v -> 1, Integer::sum));
+ CteRuleConfig cteConfig = CteRuleConfig.DEFAULT
+ .withReferenceThreshold(referenceThreshold)
+ .withTableOccurrences(tableOccurrences);
+ HepProgram spoolProgram = HepProgram.builder()
+ // Use some defined match order ensuring consistent introduction of spool operators; avoids plan flakiness
+ .addMatchOrder(HepMatchOrder.DEPTH_FIRST)
+ .addRuleInstance(new TableScanToSpoolRule(cteConfig))
+ .addRuleInstance(new RemoveInfrequentCteRule(cteConfig))
+ .build();
+ final RelNode spoolPlan = executeProgram(ctePlan, spoolProgram, mdProvider, executorProvider, cteMVs, true);
+ if (ctePlan.getRelDigest().equals(spoolPlan.getRelDigest())) {
+ return basePlan;
+ } else {
+ return spoolPlan;
+ }
+ }
+
+ /**
+ * @return whether the specified cte fulfills all preconditions for materialization.
+ */
+ private boolean isMaterializableCte(HiveRelMetadataQuery mq, RelNode cte) {
+ final boolean checkFullAggregate = conf.getBoolVar(ConfVars.HIVE_CTE_MATERIALIZE_FULL_AGGREGATE_ONLY);
+ ImmutableBitSet cteOutputColumns = ImmutableBitSet.range(cte.getRowType().getFieldCount());
+ if (checkFullAggregate && !Boolean.TRUE.equals(mq.areColumnsAggregated(cte, cteOutputColumns))) {
+ LOG.debug("Skipping CTE {} cause its not a full aggregate.", cte);
+ return false;
+ }
+ if (HiveSqlTypeUtil.containsSqlType(cte.getRowType(), SqlTypeName.NULL)) {
+ LOG.debug("Skipping CTE {} cause it contains untyped nulls", cte);
+ return false;
+ }
+ return true;
}
private boolean isMaterializedViewRewritingByTextEnabled() {
@@ -2430,13 +2521,18 @@ public class CalcitePlanner extends SemanticAnalyzer {
protected RelNode executeProgram(RelNode basePlan, HepProgram program,
RelMetadataProvider mdProvider, RexExecutor executorProvider,
- List<HiveRelOptMaterialization> materializations) {
+ List<? extends RelOptMaterialization> materializations) {
+ return executeProgram(basePlan, program, mdProvider, executorProvider, materializations, false);
+ }
+
+ private RelNode executeProgram(RelNode basePlan, HepProgram program, RelMetadataProvider mdProvider,
+ RexExecutor executorProvider, List<? extends RelOptMaterialization> materializations, boolean noDag) {
final String ruleExclusionRegex = conf.get(ConfVars.HIVE_CBO_RULE_EXCLUSION_REGEX.varname, "");
// Create planner and copy context
- HepPlanner planner = new HepPlanner(program,
- basePlan.getCluster().getPlanner().getContext());
+ HepPlanner planner =
+ new HepPlanner(program, basePlan.getCluster().getPlanner().getContext(), noDag, null, RelOptCostImpl.FACTORY);
planner.addListener(new RuleEventLogger());
List<RelMetadataProvider> list = Lists.newArrayList();
list.add(mdProvider);
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java b/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
index 4f0decbcf54..0c668b93f6f 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
@@ -437,7 +437,7 @@ public class SemanticAnalyzer extends BaseSemanticAnalyzer {
/*
* Used to check recursive CTE invocations. Similar to viewsExpanded
*/
- private List<String> ctesExpanded;
+ private List<String> ctesExpanded = new ArrayList<>();
/*
* Whether root tasks after materialized CTE linkage have been resolved
@@ -567,7 +567,7 @@ public class SemanticAnalyzer extends BaseSemanticAnalyzer {
materializedViewUpdateDesc = null;
viewsExpanded = null;
viewSelect = null;
- ctesExpanded = null;
+ ctesExpanded.clear();
globalLimitCtx.disableOpt();
viewAliasToInput.clear();
reduceSinkOperatorsAddedByEnforceBucketingSorting.clear();
@@ -2332,7 +2332,7 @@ public class SemanticAnalyzer extends BaseSemanticAnalyzer {
getMetaData(qb, false);
}
- private void getMetaData(QB qb, boolean enableMaterialization) throws SemanticException {
+ void getMetaData(QB qb, boolean enableMaterialization) throws SemanticException {
try {
Map<String, CTEClause> materializationAliasToCTEs = null;
if (enableMaterialization) {
@@ -12931,7 +12931,6 @@ public class SemanticAnalyzer extends BaseSemanticAnalyzer {
ASTNode child = ast;
this.ast = ast;
viewsExpanded = new ArrayList<String>();
- ctesExpanded = new ArrayList<String>();
// 1. analyze and process the position alias
// step processPositionAlias out of genResolvedParseTree
diff --git a/ql/src/test/queries/clientpositive/cte_cbo_plan_json.q b/ql/src/test/queries/clientpositive/cte_cbo_plan_json.q
new file mode 100644
index 00000000000..f5821b01d21
--- /dev/null
+++ b/ql/src/test/queries/clientpositive/cte_cbo_plan_json.q
@@ -0,0 +1,16 @@
+CREATE TABLE emps
+(
+ empid INTEGER,
+ deptno INTEGER,
+ name VARCHAR(10),
+ salary DECIMAL(8, 2)
+);
+
+set hive.optimize.cte.materialize.threshold=1;
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionIdentitySuggester;
+set hive.optimize.cte.materialize.full.aggregate.only=false;
+
+EXPLAIN FORMATTED CBO
+SELECT name FROM emps e WHERE salary > 50000
+UNION
+SELECT name FROM emps e WHERE salary > 50000;
diff --git a/ql/src/test/queries/clientpositive/cte_cbo_rewrite_0.q b/ql/src/test/queries/clientpositive/cte_cbo_rewrite_0.q
new file mode 100644
index 00000000000..f24b1428e8b
--- /dev/null
+++ b/ql/src/test/queries/clientpositive/cte_cbo_rewrite_0.q
@@ -0,0 +1,161 @@
+CREATE TABLE emps
+(
+ empid INTEGER,
+ deptno INTEGER,
+ name VARCHAR(10),
+ salary DECIMAL(8, 2)
+);
+
+INSERT INTO emps VALUES (0, 0, 'Rob', 100000), (1, 0, 'Alice', 110000), (2, 0, 'Mark', 120000),
+ (3, 1, 'Greg', 80000), (4, 1, 'Josh', 90000), (5, 1, 'Hector', 95000),
+ (6, 2, 'Fred', 200000), (7, 2, 'Todd', 250000);
+
+CREATE TABLE depts
+(
+ deptno INTEGER,
+ name VARCHAR(20)
+);
+
+INSERT INTO depts VALUES (0, 'Engineering'), (1, 'Support'), (2, 'Sales');
+
+-- Classify and return departments (the names) based on the average employee salary (HIGH > 100K, LOW < 100K)
+SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000;
+
+EXPLAIN
+SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000;
+
+EXPLAIN CBO
+SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000;
+
+set hive.optimize.cte.materialize.threshold=1;
+set hive.optimize.cte.materialize.full.aggregate.only=false;
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionIdentitySuggester;
+
+SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000;
+
+EXPLAIN
+SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000;
+
+EXPLAIN CBO
+SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000;
+
+-- Ensure that branch pruning (e.q., in UNION) is done before CTE detection and does not lead to broken queries/plans.
+-- A scan over a CTE should always have the respective Spool operator in the plan; in other words if we introduce CTEs
+-- we should not prune out the Spools.
+EXPLAIN CBO
+SELECT * FROM (
+SELECT d.name, 'HIGH' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000) summary
+WHERE salary_range = 'HIGH';
+
+SELECT * FROM (
+SELECT d.name, 'HIGH' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000) summary
+WHERE salary_range = 'HIGH';
+
+EXPLAIN CBO
+SELECT * FROM (
+SELECT d.name, 'HIGH' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000) summary
+WHERE salary_range = 'LOW';
+
+SELECT * FROM (
+SELECT d.name, 'HIGH' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000) summary
+WHERE salary_range = 'LOW';
\ No newline at end of file
diff --git a/ql/src/test/queries/clientpositive/cte_mat_12.q b/ql/src/test/queries/clientpositive/cte_mat_12.q
new file mode 100644
index 00000000000..89698ca1854
--- /dev/null
+++ b/ql/src/test/queries/clientpositive/cte_mat_12.q
@@ -0,0 +1,45 @@
+-- Verify that hive.optimize.cte.materialize.full.aggregate.only behaves as expected for implicitly discovered (hive.optimize.cte.suggester.class) CTEs in the query
+CREATE TABLE emps
+(
+ empid INTEGER,
+ deptno INTEGER,
+ name VARCHAR(10),
+ salary DECIMAL(8, 2)
+);
+
+set hive.optimize.cte.materialize.threshold=1;
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionIdentitySuggester;
+
+set hive.optimize.cte.materialize.full.aggregate.only=true;
+
+EXPLAIN CBO
+SELECT COUNT(*) FROM emps e GROUP BY e.deptno
+UNION
+SELECT COUNT(*) FROM emps e GROUP BY e.deptno;
+
+EXPLAIN CBO
+SELECT COUNT(*), 'A' FROM emps e GROUP BY e.deptno
+UNION
+SELECT COUNT(*), 'B' FROM emps e GROUP BY e.deptno;
+
+EXPLAIN CBO
+SELECT name FROM emps e WHERE salary > 50000
+UNION
+SELECT name FROM emps e WHERE salary > 50000;
+
+set hive.optimize.cte.materialize.full.aggregate.only=false;
+
+EXPLAIN CBO
+SELECT COUNT(*) FROM emps e GROUP BY e.deptno
+UNION
+SELECT COUNT(*) FROM emps e GROUP BY e.deptno;
+
+EXPLAIN CBO
+SELECT COUNT(*), 'A' FROM emps e GROUP BY e.deptno
+UNION
+SELECT COUNT(*), 'B' FROM emps e GROUP BY e.deptno;
+
+EXPLAIN CBO
+SELECT name FROM emps e WHERE salary > 50000
+UNION
+SELECT name FROM emps e WHERE salary > 50000;
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query1.q b/ql/src/test/queries/clientpositive/perf/cbo_query1.q
index 7cb0cd20b76..5422e3c89d1 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query1.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query1.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query1.tpl and seed 2031708268
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query10.q b/ql/src/test/queries/clientpositive/perf/cbo_query10.q
index fbdc9db6ddc..55b69c36711 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query10.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query10.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query10.tpl and seed 797269820
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query11.q b/ql/src/test/queries/clientpositive/perf/cbo_query11.q
index 224a80886b5..5dc60fb32a9 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query11.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query11.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query11.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query12.q b/ql/src/test/queries/clientpositive/perf/cbo_query12.q
index 41029cfa59d..148c434c585 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query12.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query12.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query12.tpl and seed 345591136
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query13.q b/ql/src/test/queries/clientpositive/perf/cbo_query13.q
index 72eb08d499a..5f4ff6e63e9 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query13.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query13.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query13.tpl and seed 622697896
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query14.q b/ql/src/test/queries/clientpositive/perf/cbo_query14.q
index 60123b2c32a..870ff909ad2 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query14.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query14.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
--! qt:disabled:HIVE-24167
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query14.tpl and seed 1819994127
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query15.q b/ql/src/test/queries/clientpositive/perf/cbo_query15.q
index 3beea4ea13c..39e82e03932 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query15.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query15.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query15.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query16.q b/ql/src/test/queries/clientpositive/perf/cbo_query16.q
index 5ce0b3dfb81..6a2f7fcc0bb 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query16.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query16.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
set hive.auto.convert.anti.join=true;
-- start query 1 in stream 0 using template query16.tpl and seed 171719422
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query17.q b/ql/src/test/queries/clientpositive/perf/cbo_query17.q
index 5bf98641ee2..92847b99682 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query17.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query17.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query17.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query18.q b/ql/src/test/queries/clientpositive/perf/cbo_query18.q
index 110bee5a3fb..716ba358760 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query18.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query18.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query18.tpl and seed 1978355063
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query19.q b/ql/src/test/queries/clientpositive/perf/cbo_query19.q
index abcec3601f2..3ede30d5e75 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query19.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query19.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query19.tpl and seed 1930872976
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query2.q b/ql/src/test/queries/clientpositive/perf/cbo_query2.q
index 9fcccbf4e4c..ec3f944ec9d 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query2.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query2.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query2.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query20.q b/ql/src/test/queries/clientpositive/perf/cbo_query20.q
index 1d361b8b362..9dd9fb4553b 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query20.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query20.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query20.tpl and seed 345591136
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query21.q b/ql/src/test/queries/clientpositive/perf/cbo_query21.q
index 90daea02c36..fac6ab56210 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query21.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query21.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query21.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query22.q b/ql/src/test/queries/clientpositive/perf/cbo_query22.q
index 14ceec4174f..e3d4608bbb4 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query22.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query22.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query22.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query23.q b/ql/src/test/queries/clientpositive/perf/cbo_query23.q
index a1c661b086f..9eefa38bcb0 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query23.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query23.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query23.tpl and seed 2031708268
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query24.q b/ql/src/test/queries/clientpositive/perf/cbo_query24.q
index 8994de7a236..bac654fd245 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query24.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query24.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query24.tpl and seed 1220860970
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query25.q b/ql/src/test/queries/clientpositive/perf/cbo_query25.q
index 9611e28f83a..97ed652a04e 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query25.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query25.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query25.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query26.q b/ql/src/test/queries/clientpositive/perf/cbo_query26.q
index 8b874c85bc2..735237d88f8 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query26.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query26.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query26.tpl and seed 1930872976
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query27.q b/ql/src/test/queries/clientpositive/perf/cbo_query27.q
index 48eaad5b542..7570fe1d8a0 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query27.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query27.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query27.tpl and seed 2017787633
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query28.q b/ql/src/test/queries/clientpositive/perf/cbo_query28.q
index ad9dacd65e7..84572c970ef 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query28.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query28.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
set hive.optimize.metadataonly=true;
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query29.q b/ql/src/test/queries/clientpositive/perf/cbo_query29.q
index ea9ec12d337..2c5caf0099e 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query29.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query29.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query29.tpl and seed 2031708268
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query3.q b/ql/src/test/queries/clientpositive/perf/cbo_query3.q
index b1bc55bfdfe..247c09088fc 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query3.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query3.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query3.tpl and seed 2031708268
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query30.q b/ql/src/test/queries/clientpositive/perf/cbo_query30.q
index bfb56294eb0..ec3751ea055 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query30.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query30.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query30.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query31.q b/ql/src/test/queries/clientpositive/perf/cbo_query31.q
index a2babbf23db..cdf1e462aa8 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query31.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query31.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query31.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query32.q b/ql/src/test/queries/clientpositive/perf/cbo_query32.q
index 4f3c7291a8e..a88710a2cb6 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query32.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query32.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query32.tpl and seed 2031708268
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query33.q b/ql/src/test/queries/clientpositive/perf/cbo_query33.q
index fc808aeddb8..dcd8c638c4f 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query33.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query33.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query33.tpl and seed 1930872976
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query34.q b/ql/src/test/queries/clientpositive/perf/cbo_query34.q
index 2b93a6144c3..362603164b8 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query34.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query34.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query34.tpl and seed 1971067816
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query35.q b/ql/src/test/queries/clientpositive/perf/cbo_query35.q
index 9e877806b22..53891b2e54d 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query35.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query35.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query35.tpl and seed 1930872976
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query36.q b/ql/src/test/queries/clientpositive/perf/cbo_query36.q
index cab89deeb19..3d9f3d0a273 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query36.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query36.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query36.tpl and seed 1544728811
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query37.q b/ql/src/test/queries/clientpositive/perf/cbo_query37.q
index f258dc0aa00..3ed68167636 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query37.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query37.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query37.tpl and seed 301843662
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query38.q b/ql/src/test/queries/clientpositive/perf/cbo_query38.q
index 55beed22ce5..032b0204de6 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query38.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query38.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query38.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query39.q b/ql/src/test/queries/clientpositive/perf/cbo_query39.q
index 9f0a2131f9b..12154337920 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query39.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query39.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query39.tpl and seed 1327317894
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query4.q b/ql/src/test/queries/clientpositive/perf/cbo_query4.q
index 8d89c06180e..4a3bdf7baea 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query4.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query4.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query4.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query40.q b/ql/src/test/queries/clientpositive/perf/cbo_query40.q
index b54ae4b39e9..fd43b629894 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query40.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query40.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query40.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query41.q b/ql/src/test/queries/clientpositive/perf/cbo_query41.q
index 43430156830..7ac56748fa3 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query41.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query41.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query41.tpl and seed 1581015815
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query42.q b/ql/src/test/queries/clientpositive/perf/cbo_query42.q
index 43fb4ee6255..2367ec19825 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query42.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query42.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query42.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query43.q b/ql/src/test/queries/clientpositive/perf/cbo_query43.q
index e4aff114bee..f8fbd29e0f3 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query43.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query43.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query43.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query44.q b/ql/src/test/queries/clientpositive/perf/cbo_query44.q
index 814a7213300..3c8ee7787f4 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query44.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query44.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query44.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query45.q b/ql/src/test/queries/clientpositive/perf/cbo_query45.q
index e1b4348b7ad..3aa8c2cc64e 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query45.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query45.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query45.tpl and seed 2031708268
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query46.q b/ql/src/test/queries/clientpositive/perf/cbo_query46.q
index 6a7a94ee6f5..439746bfa55 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query46.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query46.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query46.tpl and seed 803547492
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query47.q b/ql/src/test/queries/clientpositive/perf/cbo_query47.q
index 7348e4b6e74..3714f9ef79f 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query47.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query47.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query47.tpl and seed 2031708268
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query48.q b/ql/src/test/queries/clientpositive/perf/cbo_query48.q
index 21db45fc310..9538cbb2ae2 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query48.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query48.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query48.tpl and seed 622697896
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query49.q b/ql/src/test/queries/clientpositive/perf/cbo_query49.q
index fccf25d0009..6ae17317691 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query49.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query49.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query49.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query5.q b/ql/src/test/queries/clientpositive/perf/cbo_query5.q
index 405b2da571b..8b1b2d9913e 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query5.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query5.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query5.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query50.q b/ql/src/test/queries/clientpositive/perf/cbo_query50.q
index 1fb8051080f..68b284ac5ad 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query50.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query50.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query50.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query51.q b/ql/src/test/queries/clientpositive/perf/cbo_query51.q
index 16ba023f49f..e4928dad2a1 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query51.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query51.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query51.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query52.q b/ql/src/test/queries/clientpositive/perf/cbo_query52.q
index 222550240b7..af5c777924f 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query52.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query52.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query52.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query53.q b/ql/src/test/queries/clientpositive/perf/cbo_query53.q
index c0edd67c957..12783a0f1dd 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query53.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query53.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query53.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query54.q b/ql/src/test/queries/clientpositive/perf/cbo_query54.q
index 1e626c155d8..d973ccac9ed 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query54.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query54.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query54.tpl and seed 1930872976
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query55.q b/ql/src/test/queries/clientpositive/perf/cbo_query55.q
index 55176b2e33d..1e3d25b607c 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query55.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query55.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query55.tpl and seed 2031708268
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query56.q b/ql/src/test/queries/clientpositive/perf/cbo_query56.q
index 15dbc105f5b..2ae77c4e421 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query56.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query56.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query56.tpl and seed 1951559352
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query57.q b/ql/src/test/queries/clientpositive/perf/cbo_query57.q
index da23fd2a563..26747a9e420 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query57.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query57.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query57.tpl and seed 2031708268
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query58.q b/ql/src/test/queries/clientpositive/perf/cbo_query58.q
index bbc1b968823..75c8b57ec60 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query58.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query58.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query58.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query59.q b/ql/src/test/queries/clientpositive/perf/cbo_query59.q
index 9517f718702..c93c96e7693 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query59.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query59.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query59.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query6.q b/ql/src/test/queries/clientpositive/perf/cbo_query6.q
index d9ed084682c..d802e98812a 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query6.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query6.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.auto.convert.join=true;
set hive.tez.cartesian-product.enabled=true;
set hive.mapred.mode=nonstrict;
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query60.q b/ql/src/test/queries/clientpositive/perf/cbo_query60.q
index 71c33499ee7..9699abb675e 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query60.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query60.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query60.tpl and seed 1930872976
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query61.q b/ql/src/test/queries/clientpositive/perf/cbo_query61.q
index 83c95bffa59..a8773d7fcea 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query61.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query61.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query61.tpl and seed 1930872976
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query62.q b/ql/src/test/queries/clientpositive/perf/cbo_query62.q
index a3579049966..f8a24c96538 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query62.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query62.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
explain cbo
select substr(w_warehouse_name, 1, 20),
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query63.q b/ql/src/test/queries/clientpositive/perf/cbo_query63.q
index 0553f4c9312..e11c2836d0a 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query63.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query63.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query63.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query64.q b/ql/src/test/queries/clientpositive/perf/cbo_query64.q
index 24f3d856ed6..31a3b93fd03 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query64.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query64.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query64.tpl and seed 1220860970
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query65.q b/ql/src/test/queries/clientpositive/perf/cbo_query65.q
index e8b752dfc07..ef6c1bdc8fa 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query65.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query65.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query65.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query66.q b/ql/src/test/queries/clientpositive/perf/cbo_query66.q
index 518293554d0..b5de8c36756 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query66.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query66.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query66.tpl and seed 2042478054
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query67.q b/ql/src/test/queries/clientpositive/perf/cbo_query67.q
index 5781aacd0a6..7a7acc3a584 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query67.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query67.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query67.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query68.q b/ql/src/test/queries/clientpositive/perf/cbo_query68.q
index 520b9d77fb5..56fef548fcd 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query68.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query68.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query68.tpl and seed 803547492
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query69.q b/ql/src/test/queries/clientpositive/perf/cbo_query69.q
index c08c4f404c5..4deb2604af4 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query69.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query69.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
set hive.auto.convert.anti.join=true;
-- start query 1 in stream 0 using template query69.tpl and seed 797269820
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query7.q b/ql/src/test/queries/clientpositive/perf/cbo_query7.q
index 3fb7130fff8..f34a38035ea 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query7.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query7.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query7.tpl and seed 1930872976
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query70.q b/ql/src/test/queries/clientpositive/perf/cbo_query70.q
index 5b8b76e3150..c594f2c3eb5 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query70.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query70.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query70.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query71.q b/ql/src/test/queries/clientpositive/perf/cbo_query71.q
index e4d4df54437..2aba0991b0d 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query71.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query71.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query71.tpl and seed 2031708268
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query72.q b/ql/src/test/queries/clientpositive/perf/cbo_query72.q
index 941a643d7f0..060b64393ca 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query72.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query72.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query72.tpl and seed 2031708268
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query73.q b/ql/src/test/queries/clientpositive/perf/cbo_query73.q
index c78225efb28..68624fcb559 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query73.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query73.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query73.tpl and seed 1971067816
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query74.q b/ql/src/test/queries/clientpositive/perf/cbo_query74.q
index 526c3470882..84905ffd86b 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query74.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query74.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query74.tpl and seed 1556717815
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query75.q b/ql/src/test/queries/clientpositive/perf/cbo_query75.q
index 2d92853febf..17048c0c8a1 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query75.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query75.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query75.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query76.q b/ql/src/test/queries/clientpositive/perf/cbo_query76.q
index 9d2d89efab6..6b97041915f 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query76.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query76.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query76.tpl and seed 2031708268
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query77.q b/ql/src/test/queries/clientpositive/perf/cbo_query77.q
index 3d0f4842267..6f338a74551 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query77.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query77.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query77.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query78.q b/ql/src/test/queries/clientpositive/perf/cbo_query78.q
index dafb1e5eeae..6360cafc5ce 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query78.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query78.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
set hive.auto.convert.anti.join=true;
-- start query 1 in stream 0 using template query78.tpl and seed 1819994127
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query79.q b/ql/src/test/queries/clientpositive/perf/cbo_query79.q
index ce05a9a220c..60557cf7eaf 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query79.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query79.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query79.tpl and seed 2031708268
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query8.q b/ql/src/test/queries/clientpositive/perf/cbo_query8.q
index 71ab23418aa..bac5f5c16ca 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query8.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query8.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query8.tpl and seed 1766988859
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query80.q b/ql/src/test/queries/clientpositive/perf/cbo_query80.q
index 16b1da2b099..783132291f0 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query80.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query80.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query80.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query81.q b/ql/src/test/queries/clientpositive/perf/cbo_query81.q
index 722bc9e6f35..69aaa88db51 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query81.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query81.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query81.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query82.q b/ql/src/test/queries/clientpositive/perf/cbo_query82.q
index 932a71bf99c..bb24a7e25c0 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query82.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query82.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query82.tpl and seed 55585014
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query83.q b/ql/src/test/queries/clientpositive/perf/cbo_query83.q
index 551ea0da0d0..551b8653a3e 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query83.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query83.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query83.tpl and seed 1930872976
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query84.q b/ql/src/test/queries/clientpositive/perf/cbo_query84.q
index d36df574502..d8edcf6b220 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query84.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query84.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query84.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query85.q b/ql/src/test/queries/clientpositive/perf/cbo_query85.q
index e09137be9bd..17192ccbc3a 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query85.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query85.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query85.tpl and seed 622697896
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query86.q b/ql/src/test/queries/clientpositive/perf/cbo_query86.q
index a8e9941cd0f..2aa80adef3d 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query86.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query86.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query86.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query87.q b/ql/src/test/queries/clientpositive/perf/cbo_query87.q
index 79f8437ce73..77863d9b058 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query87.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query87.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query87.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query88.q b/ql/src/test/queries/clientpositive/perf/cbo_query88.q
index 09f95b980a5..de8978adb4d 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query88.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query88.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query88.tpl and seed 318176889
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query89.q b/ql/src/test/queries/clientpositive/perf/cbo_query89.q
index 768240419a4..9a331a68e52 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query89.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query89.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query89.tpl and seed 1719819282
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query9.q b/ql/src/test/queries/clientpositive/perf/cbo_query9.q
index 46d8868ac64..9c6bce148dc 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query9.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query9.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query9.tpl and seed 1490436826
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query90.q b/ql/src/test/queries/clientpositive/perf/cbo_query90.q
index fc75e67a8ea..1a2f46362ce 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query90.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query90.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query90.tpl and seed 2031708268
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query91.q b/ql/src/test/queries/clientpositive/perf/cbo_query91.q
index 7f1f8d81638..08ed2c0d379 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query91.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query91.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query91.tpl and seed 1930872976
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query92.q b/ql/src/test/queries/clientpositive/perf/cbo_query92.q
index 23b32eaf688..ef8936d8ffe 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query92.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query92.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query92.tpl and seed 2031708268
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query93.q b/ql/src/test/queries/clientpositive/perf/cbo_query93.q
index c6a7d12caf2..0372301b887 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query93.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query93.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query93.tpl and seed 1200409435
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query94.q b/ql/src/test/queries/clientpositive/perf/cbo_query94.q
index 59ba68f14b0..048ee35d36a 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query94.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query94.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
set hive.auto.convert.anti.join=true;
-- start query 1 in stream 0 using template query94.tpl and seed 2031708268
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query95.q b/ql/src/test/queries/clientpositive/perf/cbo_query95.q
index 58c9be50479..75e0d4d6277 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query95.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query95.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query95.tpl and seed 2031708268
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query96.q b/ql/src/test/queries/clientpositive/perf/cbo_query96.q
index 1561fe3eb3a..b5fa6275b87 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query96.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query96.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query96.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query97.q b/ql/src/test/queries/clientpositive/perf/cbo_query97.q
index cb754dd3fe9..732a4e66b41 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query97.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query97.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query97.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query98.q b/ql/src/test/queries/clientpositive/perf/cbo_query98.q
index a48f81e11f7..37c5a3ab0a7 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query98.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query98.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query98.tpl and seed 345591136
explain cbo
diff --git a/ql/src/test/queries/clientpositive/perf/cbo_query99.q b/ql/src/test/queries/clientpositive/perf/cbo_query99.q
index be098fa1c86..8cfc54600c5 100644
--- a/ql/src/test/queries/clientpositive/perf/cbo_query99.q
+++ b/ql/src/test/queries/clientpositive/perf/cbo_query99.q
@@ -1,3 +1,4 @@
+set hive.optimize.cte.suggester.class=org.apache.hadoop.hive.ql.optimizer.calcite.CommonTableExpressionPrintSuggester;
set hive.mapred.mode=nonstrict;
-- start query 1 in stream 0 using template query99.tpl and seed 1819994127
explain cbo
diff --git a/ql/src/test/results/clientpositive/llap/cte_cbo_plan_json.q.out b/ql/src/test/results/clientpositive/llap/cte_cbo_plan_json.q.out
new file mode 100644
index 00000000000..953b4682ddc
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/cte_cbo_plan_json.q.out
@@ -0,0 +1,37 @@
+PREHOOK: query: CREATE TABLE emps
+(
+ empid INTEGER,
+ deptno INTEGER,
+ name VARCHAR(10),
+ salary DECIMAL(8, 2)
+)
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@emps
+POSTHOOK: query: CREATE TABLE emps
+(
+ empid INTEGER,
+ deptno INTEGER,
+ name VARCHAR(10),
+ salary DECIMAL(8, 2)
+)
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@emps
+PREHOOK: query: EXPLAIN FORMATTED CBO
+SELECT name FROM emps e WHERE salary > 50000
+UNION
+SELECT name FROM emps e WHERE salary > 50000
+PREHOOK: type: QUERY
+PREHOOK: Input: default@cte_suggestion_0
+PREHOOK: Input: default@emps
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN FORMATTED CBO
+SELECT name FROM emps e WHERE salary > 50000
+UNION
+SELECT name FROM emps e WHERE salary > 50000
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@cte_suggestion_0
+POSTHOOK: Input: default@emps
+#### A masked pattern was here ####
+{"CBOPlan":"{\n \"rels\": [\n {\n \"id\": \"0\",\n \"relOp\": \"org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableScan\",\n \"table\": [\n \"default\",\n \"emps\"\n ],\n \"table:alias\": \"e\",\n \"inputs\": [],\n \"rowCount\": 1.0,\n \"avgRowSize\": 161.0,\n \"rowType\": [\n {\n \"type\": \"INTEGER\",\n \"nullable\": true,\n \"name\": \"empid\"\n },\n {\n [...]
diff --git a/ql/src/test/results/clientpositive/llap/cte_cbo_rewrite_0.q.out b/ql/src/test/results/clientpositive/llap/cte_cbo_rewrite_0.q.out
new file mode 100644
index 00000000000..8b913421a7b
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/cte_cbo_rewrite_0.q.out
@@ -0,0 +1,883 @@
+PREHOOK: query: CREATE TABLE emps
+(
+ empid INTEGER,
+ deptno INTEGER,
+ name VARCHAR(10),
+ salary DECIMAL(8, 2)
+)
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@emps
+POSTHOOK: query: CREATE TABLE emps
+(
+ empid INTEGER,
+ deptno INTEGER,
+ name VARCHAR(10),
+ salary DECIMAL(8, 2)
+)
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@emps
+PREHOOK: query: INSERT INTO emps VALUES (0, 0, 'Rob', 100000), (1, 0, 'Alice', 110000), (2, 0, 'Mark', 120000),
+ (3, 1, 'Greg', 80000), (4, 1, 'Josh', 90000), (5, 1, 'Hector', 95000),
+ (6, 2, 'Fred', 200000), (7, 2, 'Todd', 250000)
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+PREHOOK: Output: default@emps
+POSTHOOK: query: INSERT INTO emps VALUES (0, 0, 'Rob', 100000), (1, 0, 'Alice', 110000), (2, 0, 'Mark', 120000),
+ (3, 1, 'Greg', 80000), (4, 1, 'Josh', 90000), (5, 1, 'Hector', 95000),
+ (6, 2, 'Fred', 200000), (7, 2, 'Todd', 250000)
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+POSTHOOK: Output: default@emps
+POSTHOOK: Lineage: emps.deptno SCRIPT []
+POSTHOOK: Lineage: emps.empid SCRIPT []
+POSTHOOK: Lineage: emps.name SCRIPT []
+POSTHOOK: Lineage: emps.salary SCRIPT []
+PREHOOK: query: CREATE TABLE depts
+(
+ deptno INTEGER,
+ name VARCHAR(20)
+)
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@depts
+POSTHOOK: query: CREATE TABLE depts
+(
+ deptno INTEGER,
+ name VARCHAR(20)
+)
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@depts
+PREHOOK: query: INSERT INTO depts VALUES (0, 'Engineering'), (1, 'Support'), (2, 'Sales')
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+PREHOOK: Output: default@depts
+POSTHOOK: query: INSERT INTO depts VALUES (0, 'Engineering'), (1, 'Support'), (2, 'Sales')
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+POSTHOOK: Output: default@depts
+POSTHOOK: Lineage: depts.deptno SCRIPT []
+POSTHOOK: Lineage: depts.name SCRIPT []
+PREHOOK: query: SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000
+PREHOOK: type: QUERY
+PREHOOK: Input: default@depts
+PREHOOK: Input: default@emps
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@depts
+POSTHOOK: Input: default@emps
+#### A masked pattern was here ####
+Sales HIGH
+Support LOW
+Engineering HIGH
+PREHOOK: query: EXPLAIN
+SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000
+PREHOOK: type: QUERY
+PREHOOK: Input: default@depts
+PREHOOK: Input: default@emps
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN
+SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@depts
+POSTHOOK: Input: default@emps
+#### A masked pattern was here ####
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 8 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Union 4 (CONTAINS)
+ Reducer 5 <- Union 4 (SIMPLE_EDGE)
+ Reducer 6 <- Map 1 (SIMPLE_EDGE), Map 8 (SIMPLE_EDGE)
+ Reducer 7 <- Reducer 6 (SIMPLE_EDGE), Union 4 (CONTAINS)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: e
+ filterExpr: deptno is not null (type: boolean)
+ Statistics: Num rows: 8 Data size: 928 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: deptno is not null (type: boolean)
+ Statistics: Num rows: 8 Data size: 928 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: deptno (type: int), salary (type: decimal(8,2))
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 8 Data size: 928 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ null sort order: z
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 8 Data size: 928 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: decimal(8,2))
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ null sort order: z
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 8 Data size: 928 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: decimal(8,2))
+ Execution mode: vectorized, llap
+ LLAP IO: all inputs
+ Map 8
+ Map Operator Tree:
+ TableScan
+ alias: d
+ filterExpr: deptno is not null (type: boolean)
+ Statistics: Num rows: 3 Data size: 288 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: deptno is not null (type: boolean)
+ Statistics: Num rows: 3 Data size: 288 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: deptno (type: int), name (type: varchar(20))
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 3 Data size: 288 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ null sort order: z
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 3 Data size: 288 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: varchar(20))
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ null sort order: z
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 3 Data size: 288 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: varchar(20))
+ Execution mode: vectorized, llap
+ LLAP IO: all inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col1, _col3
+ Statistics: Num rows: 8 Data size: 1632 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ aggregations: sum(_col1), count(_col1)
+ keys: _col3 (type: varchar(20))
+ minReductionHashAggr: 0.625
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 3 Data size: 636 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: varchar(20))
+ null sort order: z
+ sort order: +
+ Map-reduce partition columns: _col0 (type: varchar(20))
+ Statistics: Num rows: 3 Data size: 636 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: decimal(18,2)), _col2 (type: bigint)
+ Reducer 3
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: sum(VALUE._col0), count(VALUE._col1)
+ keys: KEY._col0 (type: varchar(20))
+ mode: mergepartial
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 3 Data size: 636 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: (CAST( (_col1 / _col2) AS decimal(12,6)) >= 100000) (type: boolean)
+ Statistics: Num rows: 1 Data size: 212 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: _col0 (type: varchar(20)), 'HIGH' (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ keys: _col0 (type: varchar(20)), _col1 (type: string)
+ minReductionHashAggr: 0.5
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: varchar(20)), _col1 (type: string)
+ null sort order: zz
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: varchar(20)), _col1 (type: string)
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: COMPLETE
+ Reducer 5
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Group By Operator
+ keys: KEY._col0 (type: varchar(20)), KEY._col1 (type: string)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Reducer 6
+ Execution mode: llap
+ Reduce Operator Tree:
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col1, _col3
+ Statistics: Num rows: 8 Data size: 1632 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ aggregations: sum(_col1), count(_col1)
+ keys: _col3 (type: varchar(20))
+ minReductionHashAggr: 0.625
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 3 Data size: 636 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: varchar(20))
+ null sort order: z
+ sort order: +
+ Map-reduce partition columns: _col0 (type: varchar(20))
+ Statistics: Num rows: 3 Data size: 636 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: decimal(18,2)), _col2 (type: bigint)
+ Reducer 7
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: sum(VALUE._col0), count(VALUE._col1)
+ keys: KEY._col0 (type: varchar(20))
+ mode: mergepartial
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 3 Data size: 636 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: (CAST( (_col1 / _col2) AS decimal(12,6)) < 100000) (type: boolean)
+ Statistics: Num rows: 1 Data size: 212 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: _col0 (type: varchar(20)), 'LOW' (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 179 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ keys: _col0 (type: varchar(20)), _col1 (type: string)
+ minReductionHashAggr: 0.5
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: varchar(20)), _col1 (type: string)
+ null sort order: zz
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: varchar(20)), _col1 (type: string)
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: COMPLETE
+ Union 4
+ Vertex: Union 4
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: EXPLAIN CBO
+SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000
+PREHOOK: type: QUERY
+PREHOOK: Input: default@depts
+PREHOOK: Input: default@emps
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN CBO
+SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@depts
+POSTHOOK: Input: default@emps
+#### A masked pattern was here ####
+CBO PLAN:
+HiveAggregate(group=[{0, 1}])
+ HiveProject(name=[$0], _o__c1=[$1])
+ HiveUnion(all=[true])
+ HiveProject(name=[$0], _o__c1=[_UTF-16LE'HIGH':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"])
+ HiveFilter(condition=[>=(CAST(/($1, $2)):DECIMAL(12, 6), 100000:DECIMAL(6, 0))])
+ HiveAggregate(group=[{3}], agg#0=[sum($1)], agg#1=[count($1)])
+ HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(deptno=[$1], salary=[$3])
+ HiveFilter(condition=[IS NOT NULL($1)])
+ HiveTableScan(table=[[default, emps]], table:alias=[e])
+ HiveProject(deptno=[$0], name=[$1])
+ HiveFilter(condition=[IS NOT NULL($0)])
+ HiveTableScan(table=[[default, depts]], table:alias=[d])
+ HiveProject(name=[$0], _o__c1=[_UTF-16LE'LOW':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"])
+ HiveFilter(condition=[<(CAST(/($1, $2)):DECIMAL(12, 6), 100000:DECIMAL(6, 0))])
+ HiveAggregate(group=[{3}], agg#0=[sum($1)], agg#1=[count($1)])
+ HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(deptno=[$1], salary=[$3])
+ HiveFilter(condition=[IS NOT NULL($1)])
+ HiveTableScan(table=[[default, emps]], table:alias=[e])
+ HiveProject(deptno=[$0], name=[$1])
+ HiveFilter(condition=[IS NOT NULL($0)])
+ HiveTableScan(table=[[default, depts]], table:alias=[d])
+
+PREHOOK: query: SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000
+PREHOOK: type: QUERY
+PREHOOK: Input: default@cte_suggestion_0
+PREHOOK: Input: default@depts
+PREHOOK: Input: default@emps
+PREHOOK: Output: database:default
+PREHOOK: Output: default@cte_suggestion_0
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@cte_suggestion_0
+POSTHOOK: Input: default@depts
+POSTHOOK: Input: default@emps
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@cte_suggestion_0
+#### A masked pattern was here ####
+Sales HIGH
+Support LOW
+Engineering HIGH
+PREHOOK: query: EXPLAIN
+SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000
+PREHOOK: type: QUERY
+PREHOOK: Input: default@cte_suggestion_0
+PREHOOK: Input: default@depts
+PREHOOK: Input: default@emps
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN
+SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@cte_suggestion_0
+POSTHOOK: Input: default@depts
+POSTHOOK: Input: default@emps
+#### A masked pattern was here ####
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-4 depends on stages: Stage-2, Stage-0
+ Stage-0 depends on stages: Stage-1
+ Stage-3 depends on stages: Stage-4
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: e
+ filterExpr: deptno is not null (type: boolean)
+ Statistics: Num rows: 8 Data size: 928 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: deptno is not null (type: boolean)
+ Statistics: Num rows: 8 Data size: 928 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: deptno (type: int), salary (type: decimal(8,2))
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 8 Data size: 928 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ null sort order: z
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 8 Data size: 928 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: decimal(8,2))
+ Execution mode: vectorized, llap
+ LLAP IO: all inputs
+ Map 4
+ Map Operator Tree:
+ TableScan
+ alias: d
+ filterExpr: deptno is not null (type: boolean)
+ Statistics: Num rows: 3 Data size: 288 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: deptno is not null (type: boolean)
+ Statistics: Num rows: 3 Data size: 288 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: deptno (type: int), name (type: varchar(20))
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 3 Data size: 288 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ null sort order: z
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 3 Data size: 288 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: varchar(20))
+ Execution mode: vectorized, llap
+ LLAP IO: all inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col1, _col3
+ Statistics: Num rows: 8 Data size: 1632 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ aggregations: sum(_col1), count(_col1)
+ keys: _col3 (type: varchar(20))
+ minReductionHashAggr: 0.625
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 3 Data size: 636 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: varchar(20))
+ null sort order: z
+ sort order: +
+ Map-reduce partition columns: _col0 (type: varchar(20))
+ Statistics: Num rows: 3 Data size: 636 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: decimal(18,2)), _col2 (type: bigint)
+ Reducer 3
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: sum(VALUE._col0), count(VALUE._col1)
+ keys: KEY._col0 (type: varchar(20))
+ mode: mergepartial
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 3 Data size: 636 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 3 Data size: 636 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.cte_suggestion_0
+
+ Stage: Stage-2
+ Dependency Collection
+
+ Stage: Stage-4
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Map 5 <- Union 6 (CONTAINS)
+ Map 8 <- Union 6 (CONTAINS)
+ Reducer 7 <- Union 6 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 5
+ Map Operator Tree:
+ TableScan
+ alias: cte_suggestion_0
+ filterExpr: (CAST( ($f1 / $f2) AS decimal(12,6)) >= 100000) (type: boolean)
+ Statistics: Num rows: 3 Data size: 636 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: (CAST( ($f1 / $f2) AS decimal(12,6)) >= 100000) (type: boolean)
+ Statistics: Num rows: 1 Data size: 212 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: name (type: varchar(20)), 'HIGH' (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ keys: _col0 (type: varchar(20)), _col1 (type: string)
+ minReductionHashAggr: 0.5
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: varchar(20)), _col1 (type: string)
+ null sort order: zz
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: varchar(20)), _col1 (type: string)
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: COMPLETE
+ Execution mode: vectorized, llap
+ LLAP IO: all inputs
+ Map 8
+ Map Operator Tree:
+ TableScan
+ alias: cte_suggestion_0
+ filterExpr: (CAST( ($f1 / $f2) AS decimal(12,6)) < 100000) (type: boolean)
+ Statistics: Num rows: 3 Data size: 636 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: (CAST( ($f1 / $f2) AS decimal(12,6)) < 100000) (type: boolean)
+ Statistics: Num rows: 1 Data size: 212 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: name (type: varchar(20)), 'LOW' (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 179 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ keys: _col0 (type: varchar(20)), _col1 (type: string)
+ minReductionHashAggr: 0.5
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: varchar(20)), _col1 (type: string)
+ null sort order: zz
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: varchar(20)), _col1 (type: string)
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: COMPLETE
+ Execution mode: vectorized, llap
+ LLAP IO: all inputs
+ Reducer 7
+ Execution mode: vectorized, llap
+ Reduce Operator Tree:
+ Group By Operator
+ keys: KEY._col0 (type: varchar(20)), KEY._col1 (type: string)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Union 6
+ Vertex: Union 6
+
+ Stage: Stage-0
+ Move Operator
+ files:
+ hdfs directory: true
+#### A masked pattern was here ####
+
+ Stage: Stage-3
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: EXPLAIN CBO
+SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000
+PREHOOK: type: QUERY
+PREHOOK: Input: default@cte_suggestion_0
+PREHOOK: Input: default@depts
+PREHOOK: Input: default@emps
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN CBO
+SELECT d.name, 'HIGH'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW'
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@cte_suggestion_0
+POSTHOOK: Input: default@depts
+POSTHOOK: Input: default@emps
+#### A masked pattern was here ####
+CBO PLAN:
+HiveAggregate(group=[{0, 1}])
+ HiveProject(name=[$0], _o__c1=[$1])
+ HiveUnion(all=[true])
+ HiveProject(name=[$0], _o__c1=[_UTF-16LE'HIGH':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"])
+ HiveFilter(condition=[>=(CAST(/($1, $2)):DECIMAL(12, 6), 100000:DECIMAL(6, 0))])
+ HiveTableSpool(table=[[cte, cte_suggestion_0]])
+ HiveProject(name=[$0], $f1=[$1], $f2=[$2])
+ HiveAggregate(group=[{3}], agg#0=[sum($1)], agg#1=[count($1)])
+ HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(deptno=[$1], salary=[$3])
+ HiveFilter(condition=[IS NOT NULL($1)])
+ HiveTableScan(table=[[default, emps]], table:alias=[e])
+ HiveProject(deptno=[$0], name=[$1])
+ HiveFilter(condition=[IS NOT NULL($0)])
+ HiveTableScan(table=[[default, depts]], table:alias=[d])
+ HiveProject(name=[$0], _o__c1=[_UTF-16LE'LOW':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"])
+ HiveFilter(condition=[<(CAST(/($1, $2)):DECIMAL(12, 6), 100000:DECIMAL(6, 0))])
+ HiveTableScan(table=[[cte, cte_suggestion_0]], table:alias=[cte_suggestion_0])
+
+PREHOOK: query: EXPLAIN CBO
+SELECT * FROM (
+SELECT d.name, 'HIGH' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000) summary
+WHERE salary_range = 'HIGH'
+PREHOOK: type: QUERY
+PREHOOK: Input: default@depts
+PREHOOK: Input: default@emps
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN CBO
+SELECT * FROM (
+SELECT d.name, 'HIGH' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000) summary
+WHERE salary_range = 'HIGH'
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@depts
+POSTHOOK: Input: default@emps
+#### A masked pattern was here ####
+CBO PLAN:
+HiveAggregate(group=[{0, 1}])
+ HiveProject(name=[$0], salary_range=[_UTF-16LE'HIGH':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"])
+ HiveFilter(condition=[>=(CAST(/($1, $2)):DECIMAL(12, 6), 100000:DECIMAL(6, 0))])
+ HiveAggregate(group=[{3}], agg#0=[sum($1)], agg#1=[count($1)])
+ HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(deptno=[$1], salary=[$3])
+ HiveFilter(condition=[IS NOT NULL($1)])
+ HiveTableScan(table=[[default, emps]], table:alias=[e])
+ HiveProject(deptno=[$0], name=[$1])
+ HiveFilter(condition=[IS NOT NULL($0)])
+ HiveTableScan(table=[[default, depts]], table:alias=[d])
+
+PREHOOK: query: SELECT * FROM (
+SELECT d.name, 'HIGH' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000) summary
+WHERE salary_range = 'HIGH'
+PREHOOK: type: QUERY
+PREHOOK: Input: default@depts
+PREHOOK: Input: default@emps
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT * FROM (
+SELECT d.name, 'HIGH' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000) summary
+WHERE salary_range = 'HIGH'
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@depts
+POSTHOOK: Input: default@emps
+#### A masked pattern was here ####
+Engineering HIGH
+Sales HIGH
+PREHOOK: query: EXPLAIN CBO
+SELECT * FROM (
+SELECT d.name, 'HIGH' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000) summary
+WHERE salary_range = 'LOW'
+PREHOOK: type: QUERY
+PREHOOK: Input: default@depts
+PREHOOK: Input: default@emps
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN CBO
+SELECT * FROM (
+SELECT d.name, 'HIGH' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000) summary
+WHERE salary_range = 'LOW'
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@depts
+POSTHOOK: Input: default@emps
+#### A masked pattern was here ####
+CBO PLAN:
+HiveAggregate(group=[{0, 1}])
+ HiveProject(name=[$0], salary_range=[_UTF-16LE'LOW':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"])
+ HiveFilter(condition=[<(CAST(/($1, $2)):DECIMAL(12, 6), 100000:DECIMAL(6, 0))])
+ HiveAggregate(group=[{3}], agg#0=[sum($1)], agg#1=[count($1)])
+ HiveJoin(condition=[=($0, $2)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(deptno=[$1], salary=[$3])
+ HiveFilter(condition=[IS NOT NULL($1)])
+ HiveTableScan(table=[[default, emps]], table:alias=[e])
+ HiveProject(deptno=[$0], name=[$1])
+ HiveFilter(condition=[IS NOT NULL($0)])
+ HiveTableScan(table=[[default, depts]], table:alias=[d])
+
+PREHOOK: query: SELECT * FROM (
+SELECT d.name, 'HIGH' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000) summary
+WHERE salary_range = 'LOW'
+PREHOOK: type: QUERY
+PREHOOK: Input: default@depts
+PREHOOK: Input: default@emps
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT * FROM (
+SELECT d.name, 'HIGH' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) >= 100000
+UNION
+SELECT d.name, 'LOW' as salary_range
+FROM emps e
+INNER JOIN depts d ON e.deptno = d.deptno
+GROUP BY d.name
+HAVING AVG(e.salary) < 100000) summary
+WHERE salary_range = 'LOW'
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@depts
+POSTHOOK: Input: default@emps
+#### A masked pattern was here ####
+Support LOW
diff --git a/ql/src/test/results/clientpositive/llap/cte_mat_12.q.out b/ql/src/test/results/clientpositive/llap/cte_mat_12.q.out
new file mode 100644
index 00000000000..051c002d6b0
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/cte_mat_12.q.out
@@ -0,0 +1,182 @@
+PREHOOK: query: CREATE TABLE emps
+(
+ empid INTEGER,
+ deptno INTEGER,
+ name VARCHAR(10),
+ salary DECIMAL(8, 2)
+)
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@emps
+POSTHOOK: query: CREATE TABLE emps
+(
+ empid INTEGER,
+ deptno INTEGER,
+ name VARCHAR(10),
+ salary DECIMAL(8, 2)
+)
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@emps
+PREHOOK: query: EXPLAIN CBO
+SELECT COUNT(*) FROM emps e GROUP BY e.deptno
+UNION
+SELECT COUNT(*) FROM emps e GROUP BY e.deptno
+PREHOOK: type: QUERY
+PREHOOK: Input: default@cte_suggestion_0
+PREHOOK: Input: default@emps
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN CBO
+SELECT COUNT(*) FROM emps e GROUP BY e.deptno
+UNION
+SELECT COUNT(*) FROM emps e GROUP BY e.deptno
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@cte_suggestion_0
+POSTHOOK: Input: default@emps
+#### A masked pattern was here ####
+CBO PLAN:
+HiveAggregate(group=[{0}])
+ HiveProject(_o__c0=[$0])
+ HiveUnion(all=[true])
+ HiveProject(_o__c0=[$0])
+ HiveTableSpool(table=[[cte, cte_suggestion_0]])
+ HiveProject(_o__c0=[$1])
+ HiveAggregate(group=[{1}], agg#0=[count()])
+ HiveTableScan(table=[[default, emps]], table:alias=[e])
+ HiveProject(_o__c0=[$0])
+ HiveTableScan(table=[[cte, cte_suggestion_0]], table:alias=[cte_suggestion_0])
+
+PREHOOK: query: EXPLAIN CBO
+SELECT COUNT(*), 'A' FROM emps e GROUP BY e.deptno
+UNION
+SELECT COUNT(*), 'B' FROM emps e GROUP BY e.deptno
+PREHOOK: type: QUERY
+PREHOOK: Input: default@emps
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN CBO
+SELECT COUNT(*), 'A' FROM emps e GROUP BY e.deptno
+UNION
+SELECT COUNT(*), 'B' FROM emps e GROUP BY e.deptno
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@emps
+#### A masked pattern was here ####
+CBO PLAN:
+HiveAggregate(group=[{0, 1}])
+ HiveProject(_o__c0=[$0], _o__c1=[$1])
+ HiveUnion(all=[true])
+ HiveProject(_o__c0=[$1], _o__c1=[_UTF-16LE'A':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"])
+ HiveAggregate(group=[{1}], agg#0=[count()])
+ HiveTableScan(table=[[default, emps]], table:alias=[e])
+ HiveProject(_o__c0=[$1], _o__c1=[_UTF-16LE'B':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"])
+ HiveAggregate(group=[{1}], agg#0=[count()])
+ HiveTableScan(table=[[default, emps]], table:alias=[e])
+
+PREHOOK: query: EXPLAIN CBO
+SELECT name FROM emps e WHERE salary > 50000
+UNION
+SELECT name FROM emps e WHERE salary > 50000
+PREHOOK: type: QUERY
+PREHOOK: Input: default@emps
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN CBO
+SELECT name FROM emps e WHERE salary > 50000
+UNION
+SELECT name FROM emps e WHERE salary > 50000
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@emps
+#### A masked pattern was here ####
+CBO PLAN:
+HiveAggregate(group=[{0}])
+ HiveProject(name=[$0])
+ HiveUnion(all=[true])
+ HiveProject(name=[$2])
+ HiveFilter(condition=[>($3, 50000:DECIMAL(5, 0))])
+ HiveTableScan(table=[[default, emps]], table:alias=[e])
+ HiveProject(name=[$2])
+ HiveFilter(condition=[>($3, 50000:DECIMAL(5, 0))])
+ HiveTableScan(table=[[default, emps]], table:alias=[e])
+
+PREHOOK: query: EXPLAIN CBO
+SELECT COUNT(*) FROM emps e GROUP BY e.deptno
+UNION
+SELECT COUNT(*) FROM emps e GROUP BY e.deptno
+PREHOOK: type: QUERY
+PREHOOK: Input: default@cte_suggestion_0
+PREHOOK: Input: default@emps
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN CBO
+SELECT COUNT(*) FROM emps e GROUP BY e.deptno
+UNION
+SELECT COUNT(*) FROM emps e GROUP BY e.deptno
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@cte_suggestion_0
+POSTHOOK: Input: default@emps
+#### A masked pattern was here ####
+CBO PLAN:
+HiveAggregate(group=[{0}])
+ HiveProject(_o__c0=[$0])
+ HiveUnion(all=[true])
+ HiveProject(_o__c0=[$0])
+ HiveTableSpool(table=[[cte, cte_suggestion_0]])
+ HiveProject(_o__c0=[$1])
+ HiveAggregate(group=[{1}], agg#0=[count()])
+ HiveTableScan(table=[[default, emps]], table:alias=[e])
+ HiveProject(_o__c0=[$0])
+ HiveTableScan(table=[[cte, cte_suggestion_0]], table:alias=[cte_suggestion_0])
+
+PREHOOK: query: EXPLAIN CBO
+SELECT COUNT(*), 'A' FROM emps e GROUP BY e.deptno
+UNION
+SELECT COUNT(*), 'B' FROM emps e GROUP BY e.deptno
+PREHOOK: type: QUERY
+PREHOOK: Input: default@cte_suggestion_0
+PREHOOK: Input: default@emps
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN CBO
+SELECT COUNT(*), 'A' FROM emps e GROUP BY e.deptno
+UNION
+SELECT COUNT(*), 'B' FROM emps e GROUP BY e.deptno
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@cte_suggestion_0
+POSTHOOK: Input: default@emps
+#### A masked pattern was here ####
+CBO PLAN:
+HiveAggregate(group=[{0, 1}])
+ HiveProject(_o__c0=[$0], _o__c1=[$1])
+ HiveUnion(all=[true])
+ HiveProject(_o__c0=[$1], _o__c1=[_UTF-16LE'A':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"])
+ HiveTableSpool(table=[[cte, cte_suggestion_0]])
+ HiveProject(deptno=[$0], $f1=[$1])
+ HiveAggregate(group=[{1}], agg#0=[count()])
+ HiveTableScan(table=[[default, emps]], table:alias=[e])
+ HiveProject(_o__c0=[$1], _o__c1=[_UTF-16LE'B':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"])
+ HiveTableScan(table=[[cte, cte_suggestion_0]], table:alias=[cte_suggestion_0])
+
+PREHOOK: query: EXPLAIN CBO
+SELECT name FROM emps e WHERE salary > 50000
+UNION
+SELECT name FROM emps e WHERE salary > 50000
+PREHOOK: type: QUERY
+PREHOOK: Input: default@cte_suggestion_0
+PREHOOK: Input: default@emps
+#### A masked pattern was here ####
+POSTHOOK: query: EXPLAIN CBO
+SELECT name FROM emps e WHERE salary > 50000
+UNION
+SELECT name FROM emps e WHERE salary > 50000
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@cte_suggestion_0
+POSTHOOK: Input: default@emps
+#### A masked pattern was here ####
+CBO PLAN:
+HiveAggregate(group=[{0}])
+ HiveProject(name=[$0])
+ HiveUnion(all=[true])
+ HiveProject(name=[$0])
+ HiveTableSpool(table=[[cte, cte_suggestion_0]])
+ HiveProject(name=[$2])
+ HiveFilter(condition=[>($3, 50000:DECIMAL(5, 0))])
+ HiveTableScan(table=[[default, emps]], table:alias=[e])
+ HiveProject(name=[$0])
+ HiveTableScan(table=[[cte, cte_suggestion_0]], table:alias=[cte_suggestion_0])
+
diff --git a/ql/src/test/results/clientpositive/llap/partition_explain_ddl.q.out b/ql/src/test/results/clientpositive/llap/partition_explain_ddl.q.out
index 1be1cc023b3..7368ca8c16b 100644
--- a/ql/src/test/results/clientpositive/llap/partition_explain_ddl.q.out
+++ b/ql/src/test/results/clientpositive/llap/partition_explain_ddl.q.out
@@ -900,66 +900,66 @@ ALTER TABLE `db_bdpbase`.`default_partition_test` ADD IF NOT EXISTS PARTITION (`
ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Tennis') UPDATE STATISTICS SET('numRows'='16','rawDataSize'='635' );
-- ALTER TABLE `db_bdpbase`.`default_partition_test` ADD IF NOT EXISTS PARTITION (`sports`='__HIVE_DEFAULT_PARTITION__');
-- ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='__HIVE_DEFAULT_PARTITION__') UPDATE STATISTICS SET('numRows'='6','rawDataSize'='247' );
-ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Tennis') UPDATE STATISTICS FOR COLUMN `lastname` SET('avgColLen'='6.625','maxColLen'='10','numNulls'='0','numDVs'='16' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Tennis' FOR COLUMN `lastname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBAQw9jy4wbApZ6a/v////8B/sPU3fv/////AcCHp9kGgN/fv/r/////AYLFxPn+/////wGBkrK3Av2Ry8cCgc676Pv/////Ab/A89YBxOTiEvzixO/+/////wHD55ygA77R9+T6/////wHFq+mVB7vOpY39/////wE=
-ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Tennis') UPDATE STATISTICS FOR COLUMN `id` SET('lowValue'='1007','highValue'='1086','numNulls'='0','numDVs'='16' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Tennis' FOR COLUMN `id` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBAQgsirL7+NuZgFgJSyrf7/////AcDw1/3//////wGAw9/iAoGurff//////wH//tG+/v////8BhMDytf//////Abya/vb+/////wHCpbPM//////8B/v+D1v7/////AcD2p9r//////wGApJdwwLKYgQTByo7U//////8BwdS26vv/////AQ==
-ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Tennis') UPDATE STATISTICS FOR COLUMN `firstname` SET('avgColLen'='5.875','maxColLen'='10','numNulls'='0','numDVs'='16' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Tennis' FOR COLUMN `firstname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBAQgp+Lswf/6/Gw+/////8BgM2YkAPB0+LF/f////8B/8ulC8DngZ0CgJ+o9v//////AcDR9AWBwp7N+v////8BwZT4/QH/qIfj//////8BgP+t//3/////Ab+a5YgDxoiV3QK8+K/t/v////8Bgdruk/7/////AQ==
ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Tennis') UPDATE STATISTICS FOR COLUMN `country` SET('avgColLen'='9.5625','maxColLen'='27','numNulls'='0','numDVs'='16' );
-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Tennis' FOR COLUMN `country` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBAQweClxwTAto6P/P////8Bg5qv8gO9nZOuAYWnjJT8/////wH7y9SkAoD13gGBtJOV/v////8BgqrYtf7/////Af3z25EEgbjb3wK/2Jud//////8BgZbmgvr/////Af+XtN4Dga/pvP//////AcH6mY8B
ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Tennis') UPDATE STATISTICS FOR COLUMN `city` SET('avgColLen'='9.5625','maxColLen'='18','numNulls'='0','numDVs'='16' );
-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Tennis' FOR COLUMN `city` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBAQgfSH0gPAg7CJAoCl9ljC0YLK//////8B/picz/7/////AcDlwtP7/////wHAl84twIbP+v//////AYCoomaApeWJBoG1lab6/////wH/sNr5AoDjqJr8/////wHBz+68BcCH7fb9/////wH/+7xK
-ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Basketball') UPDATE STATISTICS FOR COLUMN `lastname` SET('avgColLen'='6.2','maxColLen'='8','numNulls'='0','numDVs'='15' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Basketball' FOR COLUMN `lastname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoA8PgZ/tvgSA18jgAoGZwtb+/////wH/zaeIAcDWpIP//////wGC/43R+/////8B/5/ogwGDhYerAfz89K4DgL6l8v7/////AYDwwZj//////wHAlpaV/f////8BgLqSK8GJg9D9/////wHCiZiWAw==
-ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Basketball') UPDATE STATISTICS FOR COLUMN `id` SET('lowValue'='1001','highValue'='1097','numNulls'='0','numDVs'='15' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Basketball' FOR COLUMN `id` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoA8PgYCNxQOA2pic/f////8BxuPc2AP8m7G4A7+Ix+z7/////wG/xNKo/f////8BgLCJswSAxK+U/f////8BgPmo2gHE6pNr/pSPjQK+4MTY+v////8BgIiW4ASEgZT0+/////8B/OCGvQE=
-ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Basketball') UPDATE STATISTICS FOR COLUMN `firstname` SET('avgColLen'='6.466666666666667','maxColLen'='9','numNulls'='0','numDVs'='15' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Basketball' FOR COLUMN `firstname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoA8PgbilpAeC7+ssvtnsxfj/////AcCW6PAFwLLw9f7/////AcHwo4v9/////wH/nYz3AoGH1J/9/////wH/7Jv7/f////8BxNrM2wK85uvN/v////8BgJnvHMCdv9b//////wGAw7/oBcLAloD5/////wE=
+ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Tennis') UPDATE STATISTICS FOR COLUMN `lastname` SET('avgColLen'='6.625','maxColLen'='10','numNulls'='0','numDVs'='16' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Tennis' FOR COLUMN `lastname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBAQw9jy4wbApZ6a/v////8B/sPU3fv/////AcCHp9kGgN/fv/r/////AYLFxPn+/////wGBkrK3Av2Ry8cCgc676Pv/////Ab/A89YBxOTiEvzixO/+/////wHD55ygA77R9+T6/////wHFq+mVB7vOpY39/////wE=
+ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Tennis') UPDATE STATISTICS FOR COLUMN `firstname` SET('avgColLen'='5.875','maxColLen'='10','numNulls'='0','numDVs'='16' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Tennis' FOR COLUMN `firstname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBAQgp+Lswf/6/Gw+/////8BgM2YkAPB0+LF/f////8B/8ulC8DngZ0CgJ+o9v//////AcDR9AWBwp7N+v////8BwZT4/QH/qIfj//////8BgP+t//3/////Ab+a5YgDxoiV3QK8+K/t/v////8Bgdruk/7/////AQ==
+ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Tennis') UPDATE STATISTICS FOR COLUMN `id` SET('lowValue'='1007','highValue'='1086','numNulls'='0','numDVs'='16' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Tennis' FOR COLUMN `id` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBAQgsirL7+NuZgFgJSyrf7/////AcDw1/3//////wGAw9/iAoGurff//////wH//tG+/v////8BhMDytf//////Abya/vb+/////wHCpbPM//////8B/v+D1v7/////AcD2p9r//////wGApJdwwLKYgQTByo7U//////8BwdS26vv/////AQ==
ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Basketball') UPDATE STATISTICS FOR COLUMN `country` SET('avgColLen'='8.666666666666666','maxColLen'='16','numNulls'='0','numDVs'='14' );
-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Basketball' FOR COLUMN `country` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoA4OhdzCtAL8jYW5A8Ce5MX6/////wHAicoqgNKYqAGA+cYigKy/jwGB3qTH/f////8BgNG/hwLDkchH/YOnjf//////Af/knsj9/////wHA+9/oBICtjo7//////wE=
ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Basketball') UPDATE STATISTICS FOR COLUMN `city` SET('avgColLen'='9.733333333333333','maxColLen'='26','numNulls'='0','numDVs'='15' );
-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Basketball' FOR COLUMN `city` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoA8PgZjVoAbAiv9ng+yLbv3bzYP8/////wGFitRK/MrIowP/zYSG+/////8BwvzCl/7/////Af+jp8UBv+6Ctf//////AYGf/9wDv6/ZgQLCvb65+f////8BvsGr/QKAtqUo
-ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Football') UPDATE STATISTICS FOR COLUMN `lastname` SET('avgColLen'='5.6521739130434785','maxColLen'='8','numNulls'='0','numDVs'='23' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Football' FOR COLUMN `lastname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBcXgqW6lwS/7cK2AsGpjOz7/////wGCm5fW//////8BvbT96wTA5rbf/P////8Bgoq+hwO/6qXp/P////8B/8GzqP7/////AcCqq+L//////wGA5PWu/v////8Bwa37xwa/u+SQ/P////8Bwcrsrf3/////Ab+2k+ADgNahiAHAyNff/v////8BwNftnP7/////AYDDkyfB/tKCA4DswM8Bgoeop/3/////Af3g8/oB
-ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Football') UPDATE STATISTICS FOR COLUMN `id` SET('lowValue'='1006','highValue'='1100','numNulls'='0','numDVs'='23' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Football' FOR COLUMN `id` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBcXgaCXhwXA1P+S/f////8Bg93J8wG91quf/P////8Bg8OW9gG9v/KA//////8Bhbns+AW7mZtJwM2xGcCm2JD7/////wHAzv2WAsDMyob9/////wHCpeL4/v////8Bv4iY4wW/56Sb+f////8BgLbX1QHB2tTlBP+Sqp36/////wGA9fLJAcGhkpcBgIXC0QPBxYbz//////8B/uGZ+vv/////AQ==
-ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Football') UPDATE STATISTICS FOR COLUMN `firstname` SET('avgColLen'='5.782608695652174','maxColLen'='9','numNulls'='0','numDVs'='23' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Football' FOR COLUMN `firstname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBcXwbS80wPA08+OBMDEmZT6/////wHA7fi1BMLU84kBvs2zyfr/////AcDRiGfCgsLrA4DDgsb9/////wG+ipqK/v////8BwMjJowWC5Jyw//////8Bv5j19Pn/////AYDBosL//////wHA9t7fAb/QhPkDgJCxk/v/////AYKK0Qb+ra2ZAoOA5Pj//////wH9vuKJAcH15av8/////wGA1v/hAg==
+ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Basketball') UPDATE STATISTICS FOR COLUMN `lastname` SET('avgColLen'='6.2','maxColLen'='8','numNulls'='0','numDVs'='15' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Basketball' FOR COLUMN `lastname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoA8PgZ/tvgSA18jgAoGZwtb+/////wH/zaeIAcDWpIP//////wGC/43R+/////8B/5/ogwGDhYerAfz89K4DgL6l8v7/////AYDwwZj//////wHAlpaV/f////8BgLqSK8GJg9D9/////wHCiZiWAw==
+ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Basketball') UPDATE STATISTICS FOR COLUMN `firstname` SET('avgColLen'='6.466666666666667','maxColLen'='9','numNulls'='0','numDVs'='15' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Basketball' FOR COLUMN `firstname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoA8PgbilpAeC7+ssvtnsxfj/////AcCW6PAFwLLw9f7/////AcHwo4v9/////wH/nYz3AoGH1J/9/////wH/7Jv7/f////8BxNrM2wK85uvN/v////8BgJnvHMCdv9b//////wGAw7/oBcLAloD5/////wE=
+ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Basketball') UPDATE STATISTICS FOR COLUMN `id` SET('lowValue'='1001','highValue'='1097','numNulls'='0','numDVs'='15' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Basketball' FOR COLUMN `id` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoA8PgYCNxQOA2pic/f////8BxuPc2AP8m7G4A7+Ix+z7/////wG/xNKo/f////8BgLCJswSAxK+U/f////8BgPmo2gHE6pNr/pSPjQK+4MTY+v////8BgIiW4ASEgZT0+/////8B/OCGvQE=
ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Football') UPDATE STATISTICS FOR COLUMN `country` SET('avgColLen'='10.826086956521738','maxColLen'='33','numNulls'='0','numDVs'='22' );
-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Football' FOR COLUMN `country` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBYWw66T/gX/gadGwYr4n/3/////AYG57zXAkP/9+/////8BvbWmigfB/KHo+v////8BwYzq0gS/zsX3/f////8B//i05Pz/////AcG9m/4D//vz0fz/////AcCqpx3AxKqpAcCF+PACgIGGwPv/////AYSQlM0B/fqH6v//////Af+1/M38/////wGCp8si/pO8vQKC4MSEAQ==
ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Football') UPDATE STATISTICS FOR COLUMN `city` SET('avgColLen'='8.782608695652174','maxColLen'='20','numNulls'='0','numDVs'='23' );
-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Football' FOR COLUMN `city` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBcXw7/xvgW/vaexAoD6oqj6/////wHBqdfw//////8BvuPJ1P7/////AYCN2LgDwaXjsv3/////Af+IxJcGwbyujvv/////Af+ttFnBqYmkA7+Yrqb7/////wHAgr+LBIGHpJP9/////wH/iY3oAYDAj7/8/////wGAie6wAYD20A3Cw5oi/ozl0QKAl+LVAcHRpASBoNS9//////8B
--- ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='__HIVE_DEFAULT_PARTITION__') UPDATE STATISTICS FOR COLUMN `lastname` SET('avgColLen'='6.333333333333333','maxColLen'='8','numNulls'='0','numDVs'='6' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='__HIVE_DEFAULT_PARTITION__' FOR COLUMN `lastname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoAYGgbePuQfB9sjg+P////8Bw93qogW83dm9/P////8BwYbtqQK/yYTa/v////8B
--- ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='__HIVE_DEFAULT_PARTITION__') UPDATE STATISTICS FOR COLUMN `id` SET('lowValue'='1005','highValue'='1098','numNulls'='0','numDVs'='6' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='__HIVE_DEFAULT_PARTITION__' FOR COLUMN `id` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoAYGgeXMkgKAqeDgAYDOgWfAjve+/P////8Bga6jv///////Af/B69cD
--- ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='__HIVE_DEFAULT_PARTITION__') UPDATE STATISTICS FOR COLUMN `firstname` SET('avgColLen'='7.0','maxColLen'='9','numNulls'='0','numDVs'='6' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='__HIVE_DEFAULT_PARTITION__' FOR COLUMN `firstname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoAYGwe3GFsGq05QBv6XU8P7/////AYHCq/AGwJ7v2Pr/////AcG15yw=
+ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Football') UPDATE STATISTICS FOR COLUMN `lastname` SET('avgColLen'='5.6521739130434785','maxColLen'='8','numNulls'='0','numDVs'='23' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Football' FOR COLUMN `lastname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBcXgqW6lwS/7cK2AsGpjOz7/////wGCm5fW//////8BvbT96wTA5rbf/P////8Bgoq+hwO/6qXp/P////8B/8GzqP7/////AcCqq+L//////wGA5PWu/v////8Bwa37xwa/u+SQ/P////8Bwcrsrf3/////Ab+2k+ADgNahiAHAyNff/v////8BwNftnP7/////AYDDkyfB/tKCA4DswM8Bgoeop/3/////Af3g8/oB
+ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Football') UPDATE STATISTICS FOR COLUMN `firstname` SET('avgColLen'='5.782608695652174','maxColLen'='9','numNulls'='0','numDVs'='23' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Football' FOR COLUMN `firstname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBcXwbS80wPA08+OBMDEmZT6/////wHA7fi1BMLU84kBvs2zyfr/////AcDRiGfCgsLrA4DDgsb9/////wG+ipqK/v////8BwMjJowWC5Jyw//////8Bv5j19Pn/////AYDBosL//////wHA9t7fAb/QhPkDgJCxk/v/////AYKK0Qb+ra2ZAoOA5Pj//////wH9vuKJAcH15av8/////wGA1v/hAg==
+ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Football') UPDATE STATISTICS FOR COLUMN `id` SET('lowValue'='1006','highValue'='1100','numNulls'='0','numDVs'='23' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Football' FOR COLUMN `id` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBcXgaCXhwXA1P+S/f////8Bg93J8wG91quf/P////8Bg8OW9gG9v/KA//////8Bhbns+AW7mZtJwM2xGcCm2JD7/////wHAzv2WAsDMyob9/////wHCpeL4/v////8Bv4iY4wW/56Sb+f////8BgLbX1QHB2tTlBP+Sqp36/////wGA9fLJAcGhkpcBgIXC0QPBxYbz//////8B/uGZ+vv/////AQ==
-- ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='__HIVE_DEFAULT_PARTITION__') UPDATE STATISTICS FOR COLUMN `country` SET('avgColLen'='9.333333333333334','maxColLen'='15','numNulls'='0','numDVs'='6' );
-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='__HIVE_DEFAULT_PARTITION__' FOR COLUMN `country` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoAYGwaXO9gaA4928+f////8BgoXl0AP/u9nv/v////8BwP3t9gP/wbzN/P////8B
-- ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='__HIVE_DEFAULT_PARTITION__') UPDATE STATISTICS FOR COLUMN `city` SET('avgColLen'='10.5','maxColLen'='17','numNulls'='0','numDVs'='6' );
-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='__HIVE_DEFAULT_PARTITION__' FOR COLUMN `city` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoAYGwaGlyQKAyt+3AYCs6oQBwt6jv/7/////Af/P9r4C//fig/v/////AQ==
-ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Cricket') UPDATE STATISTICS FOR COLUMN `lastname` SET('avgColLen'='6.642857142857143','maxColLen'='10','numNulls'='0','numDVs'='14' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Cricket' FOR COLUMN `lastname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoA4Oh/eB3gH6kIqEBoDm49H7/////wGBzfvyAcDCzlTC2eaU/P////8B/q3H8v3/////Af+bxdoFgJL6jPv/////AYG84W//1I1igqC6pwH+kKX+/v////8Bwr362gQ=
-ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Cricket') UPDATE STATISTICS FOR COLUMN `id` SET('lowValue'='1002','highValue'='1099','numNulls'='0','numDVs'='14' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Cricket' FOR COLUMN `id` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoA4Owp+5vgXAgIGC+/////8B/6frowTArIixAcCvrknA3Jmv//////8BgIHmBoGA38n6/////wHAhO5lgs76yAG+9dT6Ar+cjO36/////wGAx7GYBML+3Mj8/////wE=
-ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Cricket') UPDATE STATISTICS FOR COLUMN `firstname` SET('avgColLen'='5.285714285714286','maxColLen'='9','numNulls'='0','numDVs'='14' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Cricket' FOR COLUMN `firstname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoA4OwbS80wOB6fyi/f////8B/4eU6QKB+P2a/f////8Bw7z4qQT8+8KdAsKaipb9/////wH/pLfZ//////8Bg/367/3/////Af72zOj//////wG+8a95gvbxuf3/////Ab7LzcwFwNTv8Pz/////AQ==
+-- ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='__HIVE_DEFAULT_PARTITION__') UPDATE STATISTICS FOR COLUMN `lastname` SET('avgColLen'='6.333333333333333','maxColLen'='8','numNulls'='0','numDVs'='6' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='__HIVE_DEFAULT_PARTITION__' FOR COLUMN `lastname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoAYGgbePuQfB9sjg+P////8Bw93qogW83dm9/P////8BwYbtqQK/yYTa/v////8B
+-- ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='__HIVE_DEFAULT_PARTITION__') UPDATE STATISTICS FOR COLUMN `firstname` SET('avgColLen'='7.0','maxColLen'='9','numNulls'='0','numDVs'='6' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='__HIVE_DEFAULT_PARTITION__' FOR COLUMN `firstname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoAYGwe3GFsGq05QBv6XU8P7/////AYHCq/AGwJ7v2Pr/////AcG15yw=
+-- ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='__HIVE_DEFAULT_PARTITION__') UPDATE STATISTICS FOR COLUMN `id` SET('lowValue'='1005','highValue'='1098','numNulls'='0','numDVs'='6' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='__HIVE_DEFAULT_PARTITION__' FOR COLUMN `id` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoAYGgeXMkgKAqeDgAYDOgWfAjve+/P////8Bga6jv///////Af/B69cD
ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Cricket') UPDATE STATISTICS FOR COLUMN `country` SET('avgColLen'='9.714285714285714','maxColLen'='18','numNulls'='0','numDVs'='14' );
-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Cricket' FOR COLUMN `country` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoA4OwanPogfEsvOR+/////8B/M6b8AKCibP3/f////8Bvprl2gLB+fLA//////8B/7zUlv7/////AcXwxLz+/////wG7+JuG/v////8BgPWXlAHB1KGNAv/45vkDwvKSh/n/////AYSysbYC
ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Cricket') UPDATE STATISTICS FOR COLUMN `city` SET('avgColLen'='9.5','maxColLen'='18','numNulls'='0','numDVs'='14' );
-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Cricket' FOR COLUMN `city` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoA4Owv2KsweAxJ3Y+f////8Bv7aO5QaAo7XI+v////8BwM6p9/3/////AcCGkuoBwYfi6f//////Ab/3+6ICxPu9zwP90YX2//////8BgpXXv/j/////Af3c9qcEgZnkk/z/////Af/BmTE=
-ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Badminton') UPDATE STATISTICS FOR COLUMN `lastname` SET('avgColLen'='6.269230769230769','maxColLen'='9','numNulls'='0','numDVs'='26' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Badminton' FOR COLUMN `lastname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBoaw5bxWsDKgKgE/pbs5vz/////AYDw18AEguubl/3/////AcHBj579/////wH9pu6ABYKuotL+/////wH+x+S0//////8BgMj+2QLAjorG+f////8BwJm1pwLBgbeMBL/BgaD8/////wHAuJDKA4CXh93//////wHBuPXl+v////8B//SC/wWAweCB//////8BwMTBhv7/////AYC+l/cBwI3g1fr/////AYTay4cBvtbOsAL//sj+A8Hd+ZH4/////wE=
-ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Badminton') UPDATE STATISTICS FOR COLUMN `id` SET('lowValue'='1004','highValue'='1094','numNulls'='0','numDVs'='26' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Badminton' FOR COLUMN `id` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBoahd3Cdf32590G/6SLj/3/////AYCykT7Av92e/f////8BwYfGqwP/9oKWAYD4z638/////wGBwrxzv+i89wHA0Niv/f////8BgLL2rgGAstnL+/////8BguuQrAH+1vPaBMG99/D5/////wH//4P6BcDSr2WAnKXH/P////8BgKfB3P3/////AYCTgJEEgqb8+Pv/////AcCCvmq+0O7IAoDBhvb+/////wHA0dlC
-ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Badminton') UPDATE STATISTICS FOR COLUMN `firstname` SET('avgColLen'='5.615384615384615','maxColLen'='8','numNulls'='0','numDVs'='25' );
--- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Badminton' FOR COLUMN `firstname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBkZyJCAxAK5lvTQBMDlqKP5/////wHAl7CAA8CbmRzA8faZ/f////8Bgf/q/f//////Ab/LlfIGgNTwqvz/////AcDcuH6Ay4aWAoDC4QXA6L+q/f////8BwPv+XoDLzxPAzvF9waDs/Pz/////AcCQvbL+/////wHAuJKC//////8Bv8CUgAXAzNZPw+WL8vz/////AYD4gCH9tourAcPksWU=
+ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Cricket') UPDATE STATISTICS FOR COLUMN `lastname` SET('avgColLen'='6.642857142857143','maxColLen'='10','numNulls'='0','numDVs'='14' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Cricket' FOR COLUMN `lastname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoA4Oh/eB3gH6kIqEBoDm49H7/////wGBzfvyAcDCzlTC2eaU/P////8B/q3H8v3/////Af+bxdoFgJL6jPv/////AYG84W//1I1igqC6pwH+kKX+/v////8Bwr362gQ=
+ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Cricket') UPDATE STATISTICS FOR COLUMN `firstname` SET('avgColLen'='5.285714285714286','maxColLen'='9','numNulls'='0','numDVs'='14' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Cricket' FOR COLUMN `firstname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoA4OwbS80wOB6fyi/f////8B/4eU6QKB+P2a/f////8Bw7z4qQT8+8KdAsKaipb9/////wH/pLfZ//////8Bg/367/3/////Af72zOj//////wG+8a95gvbxuf3/////Ab7LzcwFwNTv8Pz/////AQ==
+ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Cricket') UPDATE STATISTICS FOR COLUMN `id` SET('lowValue'='1002','highValue'='1099','numNulls'='0','numDVs'='14' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Cricket' FOR COLUMN `id` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoA4Owp+5vgXAgIGC+/////8B/6frowTArIixAcCvrknA3Jmv//////8BgIHmBoGA38n6/////wHAhO5lgs76yAG+9dT6Ar+cjO36/////wGAx7GYBML+3Mj8/////wE=
ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Badminton') UPDATE STATISTICS FOR COLUMN `country` SET('avgColLen'='10.038461538461538','maxColLen'='44','numNulls'='0','numDVs'='26' );
-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Badminton' FOR COLUMN `country` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBoahbLBzAP85PKJ/f////8BwLe62gLD4vSXAYGN3q0B/6Xhi/r/////Ab2osqUFwqq+oQH+tdPa+v////8BgPenYoCwhkKAndeXA4Sn15/+/////wH+le6UAr/epnP/7bOp+v////8BgKHJV4DwkcACg4u68wGB85+S/f////8B/JyLowOAs7kWgYmk8fr/////Af/OrLoDgO721P//////AYGQ3sD7/////wE=
ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Badminton') UPDATE STATISTICS FOR COLUMN `city` SET('avgColLen'='7.961538461538462','maxColLen'='15','numNulls'='0','numDVs'='26' );
-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Badminton' FOR COLUMN `city` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBoagcnkoQHDmv3pA/6E55gChOm3qP//////AbzE6Yj7/////wH/jeLHAYC7qMgEgO/pvP//////AYDl7q35/////wHAnoCTBsDeqIT+/////wGArM7x//////8BwdCEL//10sL+/////wGB/Yxrv/G7V8HV1cT7/////wGA/62HA4C7sfsBwJT/rPr/////AYCFgrIBgbbW6P//////Ab707dv//////wGEkeKsBYLSvzy6mZuX+v////8B
+ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Badminton') UPDATE STATISTICS FOR COLUMN `lastname` SET('avgColLen'='6.269230769230769','maxColLen'='9','numNulls'='0','numDVs'='26' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Badminton' FOR COLUMN `lastname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBoaw5bxWsDKgKgE/pbs5vz/////AYDw18AEguubl/3/////AcHBj579/////wH9pu6ABYKuotL+/////wH+x+S0//////8BgMj+2QLAjorG+f////8BwJm1pwLBgbeMBL/BgaD8/////wHAuJDKA4CXh93//////wHBuPXl+v////8B//SC/wWAweCB//////8BwMTBhv7/////AYC+l/cBwI3g1fr/////AYTay4cBvtbOsAL//sj+A8Hd+ZH4/////wE=
+ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Badminton') UPDATE STATISTICS FOR COLUMN `firstname` SET('avgColLen'='5.615384615384615','maxColLen'='8','numNulls'='0','numDVs'='25' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Badminton' FOR COLUMN `firstname` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBkZyJCAxAK5lvTQBMDlqKP5/////wHAl7CAA8CbmRzA8faZ/f////8Bgf/q/f//////Ab/LlfIGgNTwqvz/////AcDcuH6Ay4aWAoDC4QXA6L+q/f////8BwPv+XoDLzxPAzvF9waDs/Pz/////AcCQvbL+/////wHAuJKC//////8Bv8CUgAXAzNZPw+WL8vz/////AYD4gCH9tourAcPksWU=
+ALTER TABLE `db_bdpbase`.`default_partition_test` PARTITION (`sports`='Badminton') UPDATE STATISTICS FOR COLUMN `id` SET('lowValue'='1004','highValue'='1094','numNulls'='0','numDVs'='26' );
+-- BIT VECTORS PRESENT FOR `db_bdpbase`.`default_partition_test` PARTITION `sports`='Badminton' FOR COLUMN `id` BUT THEY ARE NOT SUPPORTED YET. THE BASE64 VALUE FOR THE BITVECTOR IS SExMoBoahd3Cdf32590G/6SLj/3/////AYCykT7Av92e/f////8BwYfGqwP/9oKWAYD4z638/////wGBwrxzv+i89wHA0Niv/f////8BgLL2rgGAstnL+/////8BguuQrAH+1vPaBMG99/D5/////wH//4P6BcDSr2WAnKXH/P////8BgKfB3P3/////AYCTgJEEgqb8+Pv/////AcCCvmq+0O7IAoDBhvb+/////wHA0dlC
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query1.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query1.q.out
index 1216481c762..9c98d2820e2 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query1.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query1.q.out
@@ -1,3 +1,8 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[=($6, 2000)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveSortLimit(sort0=[$0], dir0=[ASC], fetch=[100])
HiveProject(c_customer_id=[$1])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query10.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query10.q.out
index c72f365e5c4..28aa1678574 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query10.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query10.q.out
@@ -1,3 +1,8 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[AND(=($6, 2002), BETWEEN(false, $8, 4, 7))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveSortLimit(sort0=[$0], sort1=[$1], sort2=[$2], sort3=[$4], sort4=[$6], sort5=[$8], sort6=[$10], sort7=[$12], dir0=[ASC], dir1=[ASC], dir2=[ASC], dir3=[ASC], dir4=[ASC], dir5=[ASC], dir6=[ASC], dir7=[ASC], fetch=[100])
HiveProject(cd_gender=[$0], cd_marital_status=[$1], cd_education_status=[$2], cnt1=[$8], cd_purchase_estimate=[$3], cnt2=[$8], cd_credit_rating=[$4], cnt3=[$8], cd_dep_count=[$5], cnt4=[$8], cd_dep_employed_count=[$6], cnt5=[$8], cd_dep_college_count=[$7], cnt6=[$8])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query11.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query11.q.out
index ade152ee71c..0fedb9e51bb 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query11.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query11.q.out
@@ -1,3 +1,27 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[=($1, 1999)])
+ HiveProject(d_date_sk=[$0], d_year=[$6])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[=($1, 2000)])
+ HiveProject(d_date_sk=[$0], d_year=[$6])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(ss_customer_sk=[$0], ss_sold_date_sk=[$3], -=[-($2, $1)])
+ HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($3))])
+ HiveProject(ss_customer_sk=[$2], ss_ext_discount_amt=[$13], ss_ext_list_price=[$16], ss_sold_date_sk=[$22])
+ HiveTableScan(table=[[default, store_sales]], table:alias=[store_sales])
+
+CTE Suggestion:
+HiveProject(ws_bill_customer_sk=[$0], ws_sold_date_sk=[$3], -=[-($2, $1)])
+ HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($3))])
+ HiveProject(ws_bill_customer_sk=[$3], ws_ext_discount_amt=[$21], ws_ext_list_price=[$24], ws_sold_date_sk=[$33])
+ HiveTableScan(table=[[default, web_sales]], table:alias=[web_sales])
+
CBO PLAN:
HiveSortLimit(sort0=[$0], sort1=[$1], sort2=[$2], sort3=[$3], dir0=[ASC], dir1=[ASC], dir2=[ASC], dir3=[ASC], fetch=[100])
HiveProject(customer_id=[$0], customer_first_name=[$2], customer_last_name=[$3], customer_birth_country=[$4])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query2.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query2.q.out
index f520ada083d..ffac11be6ea 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query2.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query2.q.out
@@ -1,3 +1,18 @@
+CTE Suggestion:
+HiveAggregate(group=[{0}], agg#0=[sum($1)], agg#1=[sum($2)], agg#2=[sum($3)], agg#3=[sum($4)], agg#4=[sum($5)], agg#5=[sum($6)], agg#6=[sum($7)])
+ HiveProject($f0=[$3], $f1=[CASE($4, $1, null:DECIMAL(7, 2))], $f2=[CASE($5, $1, null:DECIMAL(7, 2))], $f3=[CASE($6, $1, null:DECIMAL(7, 2))], $f4=[CASE($7, $1, null:DECIMAL(7, 2))], $f5=[CASE($8, $1, null:DECIMAL(7, 2))], $f6=[CASE($9, $1, null:DECIMAL(7, 2))], $f7=[CASE($10, $1, null:DECIMAL(7, 2))])
+ HiveJoin(condition=[=($2, $0)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveUnion(all=[true])
+ HiveProject(sold_date_sk=[$33], sales_price=[$22])
+ HiveFilter(condition=[IS NOT NULL($33)])
+ HiveTableScan(table=[[default, web_sales]], table:alias=[web_sales])
+ HiveProject(sold_date_sk=[$33], sales_price=[$22])
+ HiveFilter(condition=[IS NOT NULL($33)])
+ HiveTableScan(table=[[default, catalog_sales]], table:alias=[catalog_sales])
+ HiveProject(d_date_sk=[$0], d_week_seq=[$4], ==[=($14, _UTF-16LE'Sunday ')], =3=[=($14, _UTF-16LE'Monday ')], =4=[=($14, _UTF-16LE'Tuesday ')], =5=[=($14, _UTF-16LE'Wednesday')], =6=[=($14, _UTF-16LE'Thursday ')], =7=[=($14, _UTF-16LE'Friday ')], =8=[=($14, _UTF-16LE'Saturday ')])
+ HiveFilter(condition=[IS NOT NULL($4)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveSortLimit(sort0=[$0], dir0=[ASC])
HiveProject(d_week_seq1=[$9], _o__c1=[round(/($10, $1), 2)], _o__c2=[round(/($11, $2), 2)], _o__c3=[round(/($12, $3), 2)], _o__c4=[round(/($13, $4), 2)], _o__c5=[round(/($14, $5), 2)], _o__c6=[round(/($15, $6), 2)], _o__c7=[round(/($16, $7), 2)])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query23.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query23.q.out
index 65027cbbfa8..7603c18b0a0 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query23.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query23.q.out
@@ -1,3 +1,44 @@
+CTE Suggestion:
+HiveProject($f1=[$0])
+ HiveFilter(condition=[>($2, 4)])
+ HiveProject(i_item_sk=[$3], d_date=[$1], $f2=[$2])
+ HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveAggregate(group=[{0, 3}], agg#0=[count()])
+ HiveJoin(condition=[=($1, $2)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(ss_item_sk=[$1], ss_sold_date_sk=[$22])
+ HiveFilter(condition=[IS NOT NULL($22)])
+ HiveTableScan(table=[[default, store_sales]], table:alias=[store_sales])
+ HiveProject(d_date_sk=[$0], d_date=[$2])
+ HiveFilter(condition=[IN($6, 1999, 2000, 2001, 2002)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+ HiveProject(i_item_sk=[$0])
+ HiveTableScan(table=[[default, item]], table:alias=[item])
+
+CTE Suggestion:
+HiveProject(c_customer_sk=[$0])
+ HiveJoin(condition=[>($1, $2)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveFilter(condition=[IS NOT NULL($1)])
+ HiveAggregate(group=[{0}], agg#0=[sum($1)])
+ HiveProject(ss_customer_sk=[CAST($2):BIGINT NOT NULL], *=[*(CAST($9):DECIMAL(10, 0), $12)])
+ HiveFilter(condition=[IS NOT NULL($2)])
+ HiveTableScan(table=[[default, store_sales]], table:alias=[store_sales])
+ HiveProject(*=[*(0.95:DECIMAL(16, 6), $0)])
+ HiveFilter(condition=[IS NOT NULL($0)])
+ HiveAggregate(group=[{}], agg#0=[max($1)])
+ HiveAggregate(group=[{1}], agg#0=[sum($2)])
+ HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(ss_sold_date_sk=[$22], ss_customer_sk=[CAST($2):BIGINT NOT NULL], *=[*(CAST($9):DECIMAL(10, 0), $12)])
+ HiveFilter(condition=[AND(IS NOT NULL($2), IS NOT NULL($22))])
+ HiveTableScan(table=[[default, store_sales]], table:alias=[store_sales])
+ HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[IN($6, 1999, 2000, 2001, 2002)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0], d_year=[CAST(1999):INTEGER], d_moy=[CAST(1):INTEGER])
+ HiveFilter(condition=[AND(=($6, 1999), =($8, 1))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
Warning: Map Join MAPJOIN[318][bigTable=?] in task 'Reducer 7' is a cross product
CBO PLAN:
HiveAggregate(group=[{}], agg#0=[sum($0)])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query24.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query24.q.out
index 4ddafa0b407..28078ec1861 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query24.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query24.q.out
@@ -1,3 +1,23 @@
+CTE Suggestion:
+HiveJoin(condition=[AND(=($1, $7), =($2, $16))], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveJoin(condition=[AND(=($3, $6), =($0, $5))], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(ss_item_sk=[$1], ss_customer_sk=[$2], ss_store_sk=[$6], ss_ticket_number=[$8], ss_sales_price=[$12])
+ HiveFilter(condition=[AND(IS NOT NULL($6), IS NOT NULL($2))])
+ HiveTableScan(table=[[default, store_sales]], table:alias=[store_sales])
+ HiveProject(sr_item_sk=[$1], sr_ticket_number=[$8])
+ HiveTableScan(table=[[default, store_returns]], table:alias=[store_returns])
+ HiveJoin(condition=[AND(=($1, $5), <>($4, $8))], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(c_customer_sk=[$0], c_current_addr_sk=[$4], c_first_name=[$8], c_last_name=[$9], c_birth_country=[$14])
+ HiveFilter(condition=[IS NOT NULL($4)])
+ HiveTableScan(table=[[default, customer]], table:alias=[customer])
+ HiveJoin(condition=[=($7, $2)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(ca_address_sk=[$0], ca_state=[$8], ca_zip=[$9], UPPER=[UPPER($10)])
+ HiveFilter(condition=[IS NOT NULL($9)])
+ HiveTableScan(table=[[default, customer_address]], table:alias=[customer_address])
+ HiveProject(s_store_sk=[$0], s_store_name=[$5], s_state=[$24], s_zip=[$25])
+ HiveFilter(condition=[AND(=($10, 7), IS NOT NULL($25))])
+ HiveTableScan(table=[[default, store]], table:alias=[store])
+
Warning: Map Join MAPJOIN[331][bigTable=?] in task 'Reducer 7' is a cross product
CBO PLAN:
HiveProject(c_last_name=[$0], c_first_name=[$1], s_store_name=[$2], paid=[$3])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query30.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query30.q.out
index 5e5de8b19ec..88b12fed846 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query30.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query30.q.out
@@ -1,3 +1,13 @@
+CTE Suggestion:
+HiveProject(ca_address_sk=[$0], ca_state=[$8])
+ HiveFilter(condition=[IS NOT NULL($8)])
+ HiveTableScan(table=[[default, customer_address]], table:alias=[customer_address])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[=($6, 2002)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveSortLimit(sort0=[$0], sort1=[$1], sort2=[$2], sort3=[$3], sort4=[$4], sort5=[$5], sort6=[$6], sort7=[$7], sort8=[$8], sort9=[$9], sort10=[$10], sort11=[$11], sort12=[$12], dir0=[ASC], dir1=[ASC], dir2=[ASC], dir3=[ASC], dir4=[ASC], dir5=[ASC], dir6=[ASC], dir7=[ASC], dir8=[ASC], dir9=[ASC], dir10=[ASC], dir11=[ASC], dir12=[ASC], fetch=[100])
HiveProject(c_customer_id=[$1], c_salutation=[$3], c_first_name=[$4], c_last_name=[$5], c_preferred_cust_flag=[$6], c_birth_day=[$7], c_birth_month=[$8], c_birth_year=[$9], c_birth_country=[$10], c_login=[$11], c_email_address=[$12], c_last_review_date_sk=[$13], ctr_total_return=[$17])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query31.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query31.q.out
index cf6b0aab079..1d022471a2c 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query31.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query31.q.out
@@ -1,3 +1,33 @@
+CTE Suggestion:
+HiveProject(ca_address_sk=[$0], ca_county=[$7])
+ HiveFilter(condition=[IS NOT NULL($7)])
+ HiveTableScan(table=[[default, customer_address]], table:alias=[customer_address])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[AND(=($10, 1), =($6, 2000))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[AND(=($10, 2), =($6, 2000))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[AND(=($10, 3), =($6, 2000))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(ss_addr_sk=[$5], ss_ext_sales_price=[$14], ss_sold_date_sk=[$22])
+ HiveFilter(condition=[AND(IS NOT NULL($22), IS NOT NULL($5))])
+ HiveTableScan(table=[[default, store_sales]], table:alias=[store_sales])
+
+CTE Suggestion:
+HiveProject(ws_bill_addr_sk=[$6], ws_ext_sales_price=[$22], ws_sold_date_sk=[$33])
+ HiveFilter(condition=[AND(IS NOT NULL($33), IS NOT NULL($6))])
+ HiveTableScan(table=[[default, web_sales]], table:alias=[web_sales])
+
CBO PLAN:
HiveProject(ca_county=[$8], d_year=[CAST(2000):INTEGER], web_q1_q2_increase=[/($6, $1)], store_q1_q2_increase=[/($9, $11)], web_q2_q3_increase=[/($4, $6)], store_q2_q3_increase=[/($13, $9)])
HiveJoin(condition=[AND(=($8, $0), CASE(>($11, 0:DECIMAL(1, 0)), CASE($2, >(/($6, $1), /($9, $11)), false), false), CASE(>($9, 0:DECIMAL(1, 0)), CASE($7, >(/($4, $6), /($13, $9)), false), false))], joinType=[inner], algorithm=[none], cost=[not available])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query32.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query32.q.out
index d4afed4dfc3..acab380cc9c 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query32.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query32.q.out
@@ -1,3 +1,8 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[BETWEEN(false, CAST($2):TIMESTAMP(9), 1998-03-18 00:00:00:TIMESTAMP(9), 1998-06-16 00:00:00:TIMESTAMP(9))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveAggregate(group=[{}], agg#0=[sum($1)])
HiveJoin(condition=[AND(=($6, $3), >($1, $5))], joinType=[inner], algorithm=[none], cost=[not available])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query33.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query33.q.out
index 61b99c1ec82..9cfefefedb4 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query33.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query33.q.out
@@ -1,3 +1,23 @@
+CTE Suggestion:
+HiveProject(ca_address_sk=[$0], ca_gmt_offset=[CAST(-6:DECIMAL(5, 2)):DECIMAL(5, 2)])
+ HiveFilter(condition=[=($11, -6:DECIMAL(1, 0))])
+ HiveTableScan(table=[[default, customer_address]], table:alias=[customer_address])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0], d_year=[CAST(1999):INTEGER], d_moy=[CAST(3):INTEGER])
+ HiveFilter(condition=[AND(=($6, 1999), =($8, 3))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(i_item_sk=[$0], i_manufact_id=[$13])
+ HiveFilter(condition=[IS NOT NULL($13)])
+ HiveTableScan(table=[[default, item]], table:alias=[item])
+
+CTE Suggestion:
+HiveProject(i_manufact_id=[$13])
+ HiveFilter(condition=[AND(=($12, _UTF-16LE'Books '), IS NOT NULL($13))])
+ HiveTableScan(table=[[default, item]], table:alias=[item])
+
CBO PLAN:
HiveSortLimit(sort0=[$1], dir0=[ASC], fetch=[100])
HiveProject(i_manufact_id=[$0], $f1=[$1])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query35.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query35.q.out
index 65756346a03..02630885648 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query35.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query35.q.out
@@ -1,3 +1,8 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[AND(=($6, 1999), <($10, 4))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveProject(ca_state=[$0], cd_gender=[$1], cd_marital_status=[$2], cnt1=[$3], _o__c4=[$4], _o__c5=[$5], _o__c6=[$6], cd_dep_employed_count=[$7], cnt2=[$8], _o__c9=[$9], _o__c10=[$10], _o__c11=[$11], cd_dep_college_count=[$12], cnt3=[$13], _o__c14=[$14], _o__c15=[$15], _o__c16=[$16])
HiveSortLimit(sort0=[$0], sort1=[$1], sort2=[$2], sort3=[$17], sort4=[$7], sort5=[$12], dir0=[ASC], dir1=[ASC], dir2=[ASC], dir3=[ASC], dir4=[ASC], dir5=[ASC], fetch=[100])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query38.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query38.q.out
index 1d48ae1b6f0..635fe9a11c0 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query38.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query38.q.out
@@ -1,3 +1,8 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0], d_date=[$2])
+ HiveFilter(condition=[BETWEEN(false, $3, 1212, 1223)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveAggregate(group=[{}], agg#0=[count()])
HiveProject(c_last_name=[$0], c_first_name=[$1], d_date=[$2], $f3=[$3])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query4.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query4.q.out
index d8acd2b4921..1ae5580f27a 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query4.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query4.q.out
@@ -1,3 +1,33 @@
+CTE Suggestion:
+HiveProject(cs_bill_customer_sk=[$0], cs_sold_date_sk=[$5], /=[/(+(-(-($4, $3), $1), $2), 2:DECIMAL(10, 0))])
+ HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($5))])
+ HiveProject(cs_bill_customer_sk=[$2], cs_ext_discount_amt=[$21], cs_ext_sales_price=[$22], cs_ext_wholesale_cost=[$23], cs_ext_list_price=[$24], cs_sold_date_sk=[$33])
+ HiveTableScan(table=[[default, catalog_sales]], table:alias=[catalog_sales])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[=($1, 1999)])
+ HiveProject(d_date_sk=[$0], d_year=[$6])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[=($1, 2000)])
+ HiveProject(d_date_sk=[$0], d_year=[$6])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(ss_customer_sk=[$0], ss_sold_date_sk=[$5], /=[/(+(-(-($4, $3), $1), $2), 2:DECIMAL(10, 0))])
+ HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($5))])
+ HiveProject(ss_customer_sk=[$2], ss_ext_discount_amt=[$13], ss_ext_sales_price=[$14], ss_ext_wholesale_cost=[$15], ss_ext_list_price=[$16], ss_sold_date_sk=[$22])
+ HiveTableScan(table=[[default, store_sales]], table:alias=[store_sales])
+
+CTE Suggestion:
+HiveProject(ws_bill_customer_sk=[$0], ws_sold_date_sk=[$5], /=[/(+(-(-($4, $3), $1), $2), 2:DECIMAL(10, 0))])
+ HiveFilter(condition=[AND(IS NOT NULL($0), IS NOT NULL($5))])
+ HiveProject(ws_bill_customer_sk=[$3], ws_ext_discount_amt=[$21], ws_ext_sales_price=[$22], ws_ext_wholesale_cost=[$23], ws_ext_list_price=[$24], ws_sold_date_sk=[$33])
+ HiveTableScan(table=[[default, web_sales]], table:alias=[web_sales])
+
CBO PLAN:
HiveSortLimit(sort0=[$0], sort1=[$1], sort2=[$2], sort3=[$3], dir0=[ASC], dir1=[ASC], dir2=[ASC], dir3=[ASC], fetch=[100])
HiveProject(customer_id=[$0], customer_first_name=[$2], customer_last_name=[$3], customer_birth_country=[$4])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query44.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query44.q.out
index 69638f4bda0..3f5536ec9ba 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query44.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query44.q.out
@@ -1,3 +1,17 @@
+CTE Suggestion:
+HiveJoin(condition=[>($1, *(0.9:DECIMAL(1, 1), $2))], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject($f0=[$0], $f1=[CAST(/($1, $2)):DECIMAL(11, 6)])
+ HiveFilter(condition=[IS NOT NULL(CAST(/($1, $2)):DECIMAL(11, 6))])
+ HiveAggregate(group=[{1}], agg#0=[sum($21)], agg#1=[count($21)])
+ HiveFilter(condition=[=($6, 410)])
+ HiveTableScan(table=[[default, store_sales]], table:alias=[ss1])
+ HiveProject(rank_col=[CAST(/($1, $2)):DECIMAL(11, 6)])
+ HiveFilter(condition=[IS NOT NULL(CAST(/($1, $2)):DECIMAL(11, 6))])
+ HiveAggregate(group=[{0}], agg#0=[sum($1)], agg#1=[count($1)])
+ HiveProject($f0=[true], $f1=[$21])
+ HiveFilter(condition=[AND(=($6, 410), IS NULL($4))])
+ HiveTableScan(table=[[default, store_sales]], table:alias=[store_sales])
+
Warning: Map Join MAPJOIN[112][bigTable=?] in task 'Reducer 2' is a cross product
CBO PLAN:
HiveSortLimit(sort0=[$0], dir0=[ASC], fetch=[100])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query47.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query47.q.out
index 5a04302e5ee..4c0bd2c2630 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query47.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query47.q.out
@@ -1,3 +1,41 @@
+CTE Suggestion:
+HiveAggregate(group=[{5, 6, 8, 9, 11, 12}], agg#0=[sum($2)])
+ HiveJoin(condition=[=($0, $10)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveJoin(condition=[=($1, $7)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveJoin(condition=[=($3, $4)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(ss_item_sk=[$1], ss_store_sk=[$6], ss_sales_price=[$12], ss_sold_date_sk=[$22])
+ HiveFilter(condition=[AND(IS NOT NULL($22), IS NOT NULL($6))])
+ HiveTableScan(table=[[default, store_sales]], table:alias=[store_sales])
+ HiveProject(d_date_sk=[$0], d_year=[$6], d_moy=[$8])
+ HiveFilter(condition=[AND(IN($6, 2000, 1999, 2001), OR(=($6, 2000), IN(ROW($6, $8), ROW(1999, 12), ROW(2001, 1))))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+ HiveProject(s_store_sk=[$0], s_store_name=[$5], s_company_name=[$17])
+ HiveFilter(condition=[AND(IS NOT NULL($5), IS NOT NULL($17))])
+ HiveTableScan(table=[[default, store]], table:alias=[store])
+ HiveProject(i_item_sk=[$0], i_brand=[$8], i_category=[$12])
+ HiveFilter(condition=[AND(IS NOT NULL($12), IS NOT NULL($8))])
+ HiveTableScan(table=[[default, item]], table:alias=[item])
+
+CTE Suggestion:
+HiveFilter(condition=[IS NOT NULL($5)])
+ HiveProject((tok_table_or_col i_category)=[$5], (tok_table_or_col i_brand)=[$4], (tok_table_or_col s_store_name)=[$2], (tok_table_or_col s_company_name)=[$3], (tok_function sum (tok_table_or_col ss_sales_price))=[$6], rank_window_1=[rank() OVER (PARTITION BY $5, $4, $2, $3 ORDER BY $0 NULLS LAST, $1 NULLS LAST RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)])
+ HiveAggregate(group=[{5, 6, 8, 9, 11, 12}], agg#0=[sum($2)])
+ HiveJoin(condition=[=($0, $10)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveJoin(condition=[=($1, $7)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveJoin(condition=[=($3, $4)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(ss_item_sk=[$1], ss_store_sk=[$6], ss_sales_price=[$12], ss_sold_date_sk=[$22])
+ HiveFilter(condition=[AND(IS NOT NULL($22), IS NOT NULL($6))])
+ HiveTableScan(table=[[default, store_sales]], table:alias=[store_sales])
+ HiveProject(d_date_sk=[$0], d_year=[$6], d_moy=[$8])
+ HiveFilter(condition=[AND(IN($6, 2000, 1999, 2001), OR(=($6, 2000), IN(ROW($6, $8), ROW(1999, 12), ROW(2001, 1))))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+ HiveProject(s_store_sk=[$0], s_store_name=[$5], s_company_name=[$17])
+ HiveFilter(condition=[AND(IS NOT NULL($5), IS NOT NULL($17))])
+ HiveTableScan(table=[[default, store]], table:alias=[store])
+ HiveProject(i_item_sk=[$0], i_brand=[$8], i_category=[$12])
+ HiveFilter(condition=[AND(IS NOT NULL($12), IS NOT NULL($8))])
+ HiveTableScan(table=[[default, item]], table:alias=[item])
+
CBO PLAN:
HiveProject(i_category=[$0], d_year=[$1], d_moy=[$2], avg_monthly_sales=[$3], sum_sales=[$4], psum=[$5], nsum=[$6])
HiveSortLimit(sort0=[$7], sort1=[$2], dir0=[ASC], dir1=[ASC], fetch=[100])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query49.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query49.q.out
index b107e5a5fbe..aefffaccb0e 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query49.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query49.q.out
@@ -1,3 +1,8 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[AND(=($6, 2000), =($8, 12))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveSortLimit(sort0=[$0], sort1=[$3], sort2=[$4], dir0=[ASC], dir1=[ASC], dir2=[ASC], fetch=[100])
HiveProject(channel=[$0], item=[$1], return_ratio=[$2], return_rank=[$3], currency_rank=[$4])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query5.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query5.q.out
index 4a26c32b95d..4cfbd83774c 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query5.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query5.q.out
@@ -1,3 +1,8 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[BETWEEN(false, CAST($2):TIMESTAMP(9), 1998-08-04 00:00:00:TIMESTAMP(9), 1998-08-18 00:00:00:TIMESTAMP(9))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveSortLimit(sort0=[$0], sort1=[$1], dir0=[ASC], dir1=[ASC], fetch=[100])
HiveProject(channel=[$0], id=[$1], $f2=[$2], $f3=[$3], $f4=[$4])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query51.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query51.q.out
index 653ef85c7fa..1337a2cff38 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query51.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query51.q.out
@@ -1,3 +1,8 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0], d_date=[$2])
+ HiveFilter(condition=[BETWEEN(false, $3, 1212, 1223)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveSortLimit(sort0=[$0], sort1=[$1], dir0=[ASC], dir1=[ASC], fetch=[100])
HiveProject(item_sk=[$0], d_date=[$1], web_sales=[$2], store_sales=[$3], max_window_0=[$4], max_window_1=[$5])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query54.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query54.q.out
index 1feeb5e1d87..2ee9b7d455c 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query54.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query54.q.out
@@ -1,3 +1,11 @@
+CTE Suggestion:
+HiveFilter(condition=[AND(=($6, 1999), =($8, 3))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveFilter(condition=[AND(=($6, 1999), =($8, 3), IS NOT NULL($3))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
Warning: Map Join MAPJOIN[294][bigTable=?] in task 'Map 21' is a cross product
Warning: Map Join MAPJOIN[291][bigTable=?] in task 'Map 21' is a cross product
Warning: Map Join MAPJOIN[292][bigTable=?] in task 'Map 1' is a cross product
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query56.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query56.q.out
index d5229761f43..2de9051a58a 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query56.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query56.q.out
@@ -1,3 +1,18 @@
+CTE Suggestion:
+HiveProject(ca_address_sk=[$0], ca_gmt_offset=[CAST(-8:DECIMAL(5, 2)):DECIMAL(5, 2)])
+ HiveFilter(condition=[=($11, -8:DECIMAL(1, 0))])
+ HiveTableScan(table=[[default, customer_address]], table:alias=[customer_address])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0], d_year=[CAST(2000):INTEGER], d_moy=[CAST(1):INTEGER])
+ HiveFilter(condition=[AND(=($6, 2000), =($8, 1))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(i_item_id=[$1])
+ HiveFilter(condition=[IN($17, _UTF-16LE'orchid ', _UTF-16LE'chiffon ', _UTF-16LE'lace ')])
+ HiveTableScan(table=[[default, item]], table:alias=[item])
+
CBO PLAN:
HiveSortLimit(sort0=[$1], dir0=[ASC], fetch=[100])
HiveProject(i_item_id=[$0], $f1=[$1])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query57.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query57.q.out
index 5f639c2b148..d7b6ac9483a 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query57.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query57.q.out
@@ -1,3 +1,41 @@
+CTE Suggestion:
+HiveAggregate(group=[{5, 6, 8, 10, 11}], agg#0=[sum($2)])
+ HiveJoin(condition=[=($1, $9)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveJoin(condition=[=($7, $0)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveJoin(condition=[=($3, $4)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(cs_call_center_sk=[$10], cs_item_sk=[$14], cs_sales_price=[$20], cs_sold_date_sk=[$33])
+ HiveFilter(condition=[AND(IS NOT NULL($33), IS NOT NULL($10))])
+ HiveTableScan(table=[[default, catalog_sales]], table:alias=[catalog_sales])
+ HiveProject(d_date_sk=[$0], d_year=[$6], d_moy=[$8])
+ HiveFilter(condition=[AND(IN($6, 2000, 1999, 2001), OR(=($6, 2000), IN(ROW($6, $8), ROW(1999, 12), ROW(2001, 1))))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+ HiveProject(cc_call_center_sk=[$0], cc_name=[$6])
+ HiveFilter(condition=[IS NOT NULL($6)])
+ HiveTableScan(table=[[default, call_center]], table:alias=[call_center])
+ HiveProject(i_item_sk=[$0], i_brand=[$8], i_category=[$12])
+ HiveFilter(condition=[AND(IS NOT NULL($12), IS NOT NULL($8))])
+ HiveTableScan(table=[[default, item]], table:alias=[item])
+
+CTE Suggestion:
+HiveFilter(condition=[IS NOT NULL($4)])
+ HiveProject((tok_table_or_col i_category)=[$4], (tok_table_or_col i_brand)=[$3], (tok_table_or_col cc_name)=[$2], (tok_function sum (tok_table_or_col cs_sales_price))=[$5], rank_window_1=[rank() OVER (PARTITION BY $4, $3, $2 ORDER BY $0 NULLS LAST, $1 NULLS LAST RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)])
+ HiveAggregate(group=[{5, 6, 8, 10, 11}], agg#0=[sum($2)])
+ HiveJoin(condition=[=($1, $9)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveJoin(condition=[=($7, $0)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveJoin(condition=[=($3, $4)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(cs_call_center_sk=[$10], cs_item_sk=[$14], cs_sales_price=[$20], cs_sold_date_sk=[$33])
+ HiveFilter(condition=[AND(IS NOT NULL($33), IS NOT NULL($10))])
+ HiveTableScan(table=[[default, catalog_sales]], table:alias=[catalog_sales])
+ HiveProject(d_date_sk=[$0], d_year=[$6], d_moy=[$8])
+ HiveFilter(condition=[AND(IN($6, 2000, 1999, 2001), OR(=($6, 2000), IN(ROW($6, $8), ROW(1999, 12), ROW(2001, 1))))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+ HiveProject(cc_call_center_sk=[$0], cc_name=[$6])
+ HiveFilter(condition=[IS NOT NULL($6)])
+ HiveTableScan(table=[[default, call_center]], table:alias=[call_center])
+ HiveProject(i_item_sk=[$0], i_brand=[$8], i_category=[$12])
+ HiveFilter(condition=[AND(IS NOT NULL($12), IS NOT NULL($8))])
+ HiveTableScan(table=[[default, item]], table:alias=[item])
+
CBO PLAN:
HiveProject(i_category=[$0], i_brand=[$1], d_year=[$2], d_moy=[$3], avg_monthly_sales=[$4], sum_sales=[$5], psum=[$6], nsum=[$7])
HiveSortLimit(sort0=[$8], sort1=[$2], dir0=[ASC], dir1=[ASC], fetch=[100])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query58.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query58.q.out
index 636a2408815..6ce6d89dc74 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query58.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query58.q.out
@@ -1,3 +1,23 @@
+CTE Suggestion:
+HiveProject(d_date=[$0])
+ HiveJoin(condition=[=($1, $3)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveJoin(condition=[true], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(d_date=[$2], d_week_seq=[$4])
+ HiveFilter(condition=[AND(IS NOT NULL($4), IS NOT NULL($2))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+ HiveFilter(condition=[sq_count_check($0)])
+ HiveAggregate(group=[{}], cnt=[COUNT()])
+ HiveFilter(condition=[=($2, 1998-02-19)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+ HiveProject(d_week_seq=[$4])
+ HiveFilter(condition=[AND(=($2, 1998-02-19), IS NOT NULL($4))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0], d_date=[$2])
+ HiveFilter(condition=[IS NOT NULL($2)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
Warning: Map Join MAPJOIN[375][bigTable=?] in task 'Map 5' is a cross product
CBO PLAN:
HiveSortLimit(sort0=[$0], sort1=[$1], dir0=[ASC], dir1=[ASC], fetch=[100])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query59.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query59.q.out
index c31f3074792..560fd46a05d 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query59.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query59.q.out
@@ -1,3 +1,13 @@
+CTE Suggestion:
+HiveProject($f0=[$4], $f1=[$0], $f2=[CASE($5, $1, null:DECIMAL(7, 2))], $f3=[CASE($6, $1, null:DECIMAL(7, 2))], $f4=[CASE($7, $1, null:DECIMAL(7, 2))], $f5=[CASE($8, $1, null:DECIMAL(7, 2))], $f6=[CASE($9, $1, null:DECIMAL(7, 2))], $f7=[CASE($10, $1, null:DECIMAL(7, 2))], $f8=[CASE($11, $1, null:DECIMAL(7, 2))])
+ HiveJoin(condition=[=($3, $2)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(ss_store_sk=[$6], ss_sales_price=[$12], ss_sold_date_sk=[$22])
+ HiveFilter(condition=[AND(IS NOT NULL($22), IS NOT NULL($6))])
+ HiveTableScan(table=[[default, store_sales]], table:alias=[store_sales])
+ HiveProject(d_date_sk=[$0], d_week_seq=[$4], ==[=($14, _UTF-16LE'Sunday ')], =3=[=($14, _UTF-16LE'Monday ')], =4=[=($14, _UTF-16LE'Tuesday ')], =5=[=($14, _UTF-16LE'Wednesday')], =6=[=($14, _UTF-16LE'Thursday ')], =7=[=($14, _UTF-16LE'Friday ')], =8=[=($14, _UTF-16LE'Saturday ')])
+ HiveFilter(condition=[IS NOT NULL($4)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveSortLimit(sort0=[$0], sort1=[$1], sort2=[$2], dir0=[ASC], dir1=[ASC], dir2=[ASC], fetch=[100])
HiveProject(s_store_name1=[$2], s_store_id1=[$1], d_week_seq1=[$5], _o__c3=[/($7, $17)], _o__c4=[/($8, $18)], _o__c5=[/($9, $9)], _o__c6=[/($10, $19)], _o__c7=[/($11, $20)], _o__c8=[/($12, $21)], _o__c9=[/($13, $22)])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query60.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query60.q.out
index 3109d4d558e..64126092ba0 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query60.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query60.q.out
@@ -1,3 +1,18 @@
+CTE Suggestion:
+HiveProject(ca_address_sk=[$0], ca_gmt_offset=[CAST(-6:DECIMAL(5, 2)):DECIMAL(5, 2)])
+ HiveFilter(condition=[=($11, -6:DECIMAL(1, 0))])
+ HiveTableScan(table=[[default, customer_address]], table:alias=[customer_address])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0], d_year=[CAST(1999):INTEGER], d_moy=[CAST(9):INTEGER])
+ HiveFilter(condition=[AND(=($6, 1999), =($8, 9))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(i_item_id=[$1])
+ HiveFilter(condition=[=($12, _UTF-16LE'Children ')])
+ HiveTableScan(table=[[default, item]], table:alias=[item])
+
CBO PLAN:
HiveSortLimit(sort0=[$0], sort1=[$1], dir0=[ASC], dir1=[ASC], fetch=[100])
HiveProject(i_item_id=[$0], $f1=[$1])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query61.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query61.q.out
index de4d3233142..40d79f05883 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query61.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query61.q.out
@@ -1,3 +1,27 @@
+CTE Suggestion:
+HiveJoin(condition=[=($2, $1)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(c_customer_sk=[$0], c_current_addr_sk=[$4])
+ HiveFilter(condition=[IS NOT NULL($4)])
+ HiveTableScan(table=[[default, customer]], table:alias=[customer])
+ HiveProject(ca_address_sk=[$0])
+ HiveFilter(condition=[=($11, -7:DECIMAL(1, 0))])
+ HiveTableScan(table=[[default, customer_address]], table:alias=[customer_address])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[AND(=($6, 1999), =($8, 11))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(i_item_sk=[$0])
+ HiveFilter(condition=[=($12, _UTF-16LE'Electronics ')])
+ HiveTableScan(table=[[default, item]], table:alias=[item])
+
+CTE Suggestion:
+HiveProject(s_store_sk=[$0])
+ HiveFilter(condition=[=($27, -7:DECIMAL(1, 0))])
+ HiveTableScan(table=[[default, store]], table:alias=[store])
+
Warning: Map Join MAPJOIN[249][bigTable=?] in task 'Reducer 6' is a cross product
CBO PLAN:
HiveProject(promotions=[$0], total=[$1], _o__c2=[*(/(CAST($0):DECIMAL(15, 4), CAST($1):DECIMAL(15, 4)), 100:DECIMAL(10, 0))])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query64.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query64.q.out
index 472b98006e9..43df05c1e01 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query64.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query64.q.out
@@ -1,3 +1,42 @@
+CTE Suggestion:
+HiveJoin(condition=[=($0, $11)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(ss_item_sk=[$1], ss_customer_sk=[$2], ss_cdemo_sk=[$3], ss_hdemo_sk=[$4], ss_addr_sk=[$5], ss_store_sk=[$6], ss_ticket_number=[$8], ss_wholesale_cost=[$10], ss_list_price=[$11], ss_coupon_amt=[$18], ss_sold_date_sk=[$22])
+ HiveFilter(condition=[AND(IS NOT NULL($22), IS NOT NULL($6), IS NOT NULL($2), IS NOT NULL($3), IS NOT NULL($7), IS NOT NULL($4), IS NOT NULL($5))])
+ HiveTableScan(table=[[default, store_sales]], table:alias=[store_sales])
+ HiveProject(i_item_sk=[$0], i_product_name=[$21])
+ HiveFilter(condition=[AND(IN($17, _UTF-16LE'maroon ', _UTF-16LE'burnished ', _UTF-16LE'dim ', _UTF-16LE'steel ', _UTF-16LE'navajo ', _UTF-16LE'chocolate '), BETWEEN(false, $5, 36:DECIMAL(12, 2), 45:DECIMAL(12, 2)))])
+ HiveTableScan(table=[[default, item]], table:alias=[item])
+
+CTE Suggestion:
+HiveProject(c_customer_sk=[$0], c_current_cdemo_sk=[$2], c_current_hdemo_sk=[$3], c_current_addr_sk=[$4], c_first_shipto_date_sk=[$5], c_first_sales_date_sk=[$6])
+ HiveFilter(condition=[AND(IS NOT NULL($6), IS NOT NULL($5), IS NOT NULL($2), IS NOT NULL($3), IS NOT NULL($4))])
+ HiveTableScan(table=[[default, customer]], table:alias=[customer])
+
+CTE Suggestion:
+HiveProject(cs_item_sk=[$0])
+ HiveFilter(condition=[>($1, *(2:DECIMAL(10, 0), $2))])
+ HiveAggregate(group=[{0}], agg#0=[sum($2)], agg#1=[sum($5)])
+ HiveJoin(condition=[AND(=($0, $3), =($1, $4))], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(cs_item_sk=[$14], cs_order_number=[$16], cs_ext_list_price=[$24])
+ HiveTableScan(table=[[default, catalog_sales]], table:alias=[catalog_sales])
+ HiveProject(cr_item_sk=[$1], cr_order_number=[$15], +=[+(+($22, $23), $24)])
+ HiveTableScan(table=[[default, catalog_returns]], table:alias=[catalog_returns])
+
+CTE Suggestion:
+HiveProject(hd_demo_sk=[$0])
+ HiveFilter(condition=[IS NOT NULL($1)])
+ HiveTableScan(table=[[default, household_demographics]], table:alias=[hd1])
+
+CTE Suggestion:
+HiveProject(hd_demo_sk=[$0])
+ HiveFilter(condition=[IS NOT NULL($1)])
+ HiveTableScan(table=[[default, household_demographics]], table:alias=[hd2])
+
+CTE Suggestion:
+HiveProject(s_store_sk=[$0], s_store_name=[$5], s_zip=[$25])
+ HiveFilter(condition=[AND(IS NOT NULL($5), IS NOT NULL($25))])
+ HiveTableScan(table=[[default, store]], table:alias=[store])
+
CBO PLAN:
HiveProject(product_name=[$0], store_name=[$1], store_zip=[$2], b_street_number=[$3], b_streen_name=[$4], b_city=[$5], b_zip=[$6], c_street_number=[$7], c_street_name=[$8], c_city=[$9], c_zip=[$10], syear=[CAST(2000):INTEGER], cnt=[$11], s1=[$12], s2=[$13], s3=[$14], s11=[$15], s21=[$16], s31=[$17], syear1=[CAST(2001):INTEGER], cnt1=[$18])
HiveSortLimit(sort0=[$0], sort1=[$1], sort2=[$18], dir0=[ASC], dir1=[ASC], dir2=[ASC])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query65.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query65.q.out
index d8f2a2f16af..96980593693 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query65.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query65.q.out
@@ -1,3 +1,13 @@
+CTE Suggestion:
+HiveAggregate(group=[{0, 1}], agg#0=[sum($2)])
+ HiveJoin(condition=[=($3, $4)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(ss_item_sk=[$1], ss_store_sk=[$6], ss_sales_price=[$12], ss_sold_date_sk=[$22])
+ HiveFilter(condition=[AND(IS NOT NULL($22), IS NOT NULL($6))])
+ HiveTableScan(table=[[default, store_sales]], table:alias=[store_sales])
+ HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[BETWEEN(false, $3, 1212, 1223)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveSortLimit(sort0=[$0], sort1=[$1], dir0=[ASC], dir1=[ASC], fetch=[100])
HiveProject(s_store_name=[$1], i_item_desc=[$8], revenue=[$4], i_current_price=[$9], i_wholesale_cost=[$10], i_brand=[$11])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query66.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query66.q.out
index 30dbee621e6..571b68e79a1 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query66.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query66.q.out
@@ -1,3 +1,18 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0], ==[=($8, 1)], =2=[=($8, 2)], =3=[=($8, 3)], =4=[=($8, 4)], =5=[=($8, 5)], =6=[=($8, 6)], =7=[=($8, 7)], =8=[=($8, 8)], =9=[=($8, 9)], =10=[=($8, 10)], =11=[=($8, 11)], =12=[=($8, 12)])
+ HiveFilter(condition=[=($6, 2002)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(sm_ship_mode_sk=[$0])
+ HiveFilter(condition=[IN($4, _UTF-16LE'DIAMOND ', _UTF-16LE'AIRBORNE ')])
+ HiveTableScan(table=[[default, ship_mode]], table:alias=[ship_mode])
+
+CTE Suggestion:
+HiveProject(t_time_sk=[$0])
+ HiveFilter(condition=[BETWEEN(false, $2, 49530, 78330)])
+ HiveTableScan(table=[[default, time_dim]], table:alias=[time_dim])
+
CBO PLAN:
HiveProject(w_warehouse_name=[$0], w_warehouse_sq_ft=[$1], w_city=[$2], w_county=[$3], w_state=[$4], w_country=[$5], ship_carriers=[CAST(_UTF-16LE'DIAMOND,AIRBORNE':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"):VARCHAR(2147483647) CHARACTER SET "UTF-16LE"], year=[CAST(2002):INTEGER], jan_sales=[$6], feb_sales=[$7], mar_sales=[$8], apr_sales=[$9], may_sales=[$10], jun_sales=[$11], jul_sales=[$12], aug_sales=[$13], sep_sales=[$14], oct_sales=[$15], nov_sales=[$16], dec_sales=[$17], jan_sal [...]
HiveSortLimit(sort0=[$0], dir0=[ASC], fetch=[100])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query69.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query69.q.out
index 48c0c4e3553..9ffac55dfe0 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query69.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query69.q.out
@@ -1,3 +1,8 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[AND(=($6, 1999), BETWEEN(false, $8, 1, 3))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveSortLimit(sort0=[$0], sort1=[$1], sort2=[$2], sort3=[$4], sort4=[$6], dir0=[ASC], dir1=[ASC], dir2=[ASC], dir3=[ASC], dir4=[ASC], fetch=[100])
HiveProject(cd_gender=[$0], cd_marital_status=[$1], cd_education_status=[$2], cnt1=[$5], cd_purchase_estimate=[$3], cnt2=[$5], cd_credit_rating=[$4], cnt3=[$5])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query71.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query71.q.out
index 24057aa5d40..2cd4db166a9 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query71.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query71.q.out
@@ -1,3 +1,8 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[AND(=($8, 12), =($6, 2001))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveProject(brand_id=[$0], brand=[$1], t_hour=[$2], t_minute=[$3], ext_price=[$4])
HiveSortLimit(sort0=[$4], sort1=[$5], dir0=[DESC], dir1=[ASC])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query74.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query74.q.out
index 162ca9c8111..c4d80c2b718 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query74.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query74.q.out
@@ -1,3 +1,23 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[=($6, 1998)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[=($6, 1999)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(ss_customer_sk=[$2], ss_net_paid=[$19], ss_sold_date_sk=[$22])
+ HiveFilter(condition=[AND(IS NOT NULL($2), IS NOT NULL($22))])
+ HiveTableScan(table=[[default, store_sales]], table:alias=[store_sales])
+
+CTE Suggestion:
+HiveProject(ws_bill_customer_sk=[$3], ws_net_paid=[$28], ws_sold_date_sk=[$33])
+ HiveFilter(condition=[AND(IS NOT NULL($3), IS NOT NULL($33))])
+ HiveTableScan(table=[[default, web_sales]], table:alias=[web_sales])
+
CBO PLAN:
HiveSortLimit(sort0=[$2], sort1=[$0], sort2=[$1], dir0=[ASC], dir1=[ASC], dir2=[ASC], fetch=[100])
HiveProject(customer_id=[$0], customer_first_name=[$1], customer_last_name=[$2])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query75.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query75.q.out
index 1d9270ecd26..73ff1e22219 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query75.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query75.q.out
@@ -1,3 +1,33 @@
+CTE Suggestion:
+HiveProject(cs_item_sk=[$14], cs_order_number=[$16], cs_quantity=[$17], cs_ext_sales_price=[$22], cs_sold_date_sk=[$33])
+ HiveFilter(condition=[IS NOT NULL($33)])
+ HiveTableScan(table=[[default, catalog_sales]], table:alias=[catalog_sales])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[=($6, 2001)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[=($6, 2002)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(i_item_sk=[$0], i_brand_id=[$7], i_class_id=[$9], i_category_id=[$11], i_manufact_id=[$13])
+ HiveFilter(condition=[AND(=($12, _UTF-16LE'Sports '), IS NOT NULL($7), IS NOT NULL($9), IS NOT NULL($11), IS NOT NULL($13))])
+ HiveTableScan(table=[[default, item]], table:alias=[item])
+
+CTE Suggestion:
+HiveProject(ss_item_sk=[$1], ss_ticket_number=[$8], ss_quantity=[$9], ss_ext_sales_price=[$14], ss_sold_date_sk=[$22])
+ HiveFilter(condition=[IS NOT NULL($22)])
+ HiveTableScan(table=[[default, store_sales]], table:alias=[store_sales])
+
+CTE Suggestion:
+HiveProject(ws_item_sk=[$2], ws_order_number=[$16], ws_quantity=[$17], ws_ext_sales_price=[$22], ws_sold_date_sk=[$33])
+ HiveFilter(condition=[IS NOT NULL($33)])
+ HiveTableScan(table=[[default, web_sales]], table:alias=[web_sales])
+
CBO PLAN:
HiveProject(prev_year=[CAST(2001):INTEGER], year=[CAST(2002):INTEGER], i_brand_id=[$0], i_class_id=[$1], i_category_id=[$2], i_manufact_id=[$3], prev_yr_cnt=[$4], curr_yr_cnt=[$5], sales_cnt_diff=[$6], sales_amt_diff=[$7])
HiveSortLimit(sort0=[$6], dir0=[ASC], fetch=[100])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query77.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query77.q.out
index 353f6d06182..bb000beb3a6 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query77.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query77.q.out
@@ -1,3 +1,8 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[BETWEEN(false, CAST($2):TIMESTAMP(9), 1998-08-04 00:00:00:TIMESTAMP(9), 1998-09-03 00:00:00:TIMESTAMP(9))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
Warning: Map Join MAPJOIN[213][bigTable=?] in task 'Reducer 9' is a cross product
CBO PLAN:
HiveSortLimit(sort0=[$0], sort1=[$1], dir0=[ASC], dir1=[ASC], fetch=[100])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query78.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query78.q.out
index f1ef1809e23..e706f23160f 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query78.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query78.q.out
@@ -1,3 +1,8 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[=($6, 2000)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveProject(ss_sold_year=[CAST(2000):INTEGER], ss_item_sk=[$0], ss_customer_sk=[$1], ratio=[$2], store_qty=[$3], store_wholesale_cost=[$4], store_sales_price=[$5], other_chan_qty=[$6], other_chan_wholesale_cost=[$7], other_chan_sales_price=[$8])
HiveSortLimit(sort0=[$0], sort1=[$1], sort2=[$9], sort3=[$10], sort4=[$11], sort5=[$6], sort6=[$7], sort7=[$8], sort8=[$12], dir0=[ASC], dir1=[ASC], dir2=[DESC], dir3=[DESC], dir4=[DESC], dir5=[ASC], dir6=[ASC], dir7=[ASC], dir8=[ASC], fetch=[100])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query80.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query80.q.out
index b8fae4529e6..f8676ae0359 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query80.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query80.q.out
@@ -1,3 +1,18 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[BETWEEN(false, CAST($2):TIMESTAMP(9), 1998-08-04 00:00:00:TIMESTAMP(9), 1998-09-03 00:00:00:TIMESTAMP(9))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(i_item_sk=[$0])
+ HiveFilter(condition=[>($5, 50:DECIMAL(2, 0))])
+ HiveTableScan(table=[[default, item]], table:alias=[item])
+
+CTE Suggestion:
+HiveProject(p_promo_sk=[$0])
+ HiveFilter(condition=[=($11, _UTF-16LE'N')])
+ HiveTableScan(table=[[default, promotion]], table:alias=[promotion])
+
CBO PLAN:
HiveSortLimit(sort0=[$0], sort1=[$1], dir0=[ASC], dir1=[ASC], fetch=[100])
HiveProject(channel=[$0], id=[$1], $f2=[$2], $f3=[$3], $f4=[$4])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query81.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query81.q.out
index d1bdb18108d..a8ee1e33b08 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query81.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query81.q.out
@@ -1,3 +1,13 @@
+CTE Suggestion:
+HiveProject(ca_address_sk=[$0], ca_state=[$8])
+ HiveFilter(condition=[IS NOT NULL($8)])
+ HiveTableScan(table=[[default, customer_address]], table:alias=[customer_address])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[=($6, 1998)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveProject(c_customer_id=[$0], c_salutation=[$1], c_first_name=[$2], c_last_name=[$3], ca_street_number=[$4], ca_street_name=[$5], ca_street_type=[$6], ca_suite_number=[$7], ca_city=[$8], ca_county=[$9], ca_state=[CAST(_UTF-16LE'IL'):CHAR(2) CHARACTER SET "UTF-16LE"], ca_zip=[$10], ca_country=[$11], ca_gmt_offset=[$12], ca_location_type=[$13], ctr_total_return=[$14])
HiveSortLimit(sort0=[$0], sort1=[$1], sort2=[$2], sort3=[$3], sort4=[$4], sort5=[$5], sort6=[$6], sort7=[$7], sort8=[$8], sort9=[$9], sort10=[$10], sort11=[$11], sort12=[$12], sort13=[$13], sort14=[$14], dir0=[ASC], dir1=[ASC], dir2=[ASC], dir3=[ASC], dir4=[ASC], dir5=[ASC], dir6=[ASC], dir7=[ASC], dir8=[ASC], dir9=[ASC], dir10=[ASC], dir11=[ASC], dir12=[ASC], dir13=[ASC], dir14=[ASC], fetch=[100])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query83.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query83.q.out
index 344832bee83..e494f70f2e5 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query83.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query83.q.out
@@ -1,3 +1,18 @@
+CTE Suggestion:
+HiveProject(d_date=[$0])
+ HiveSemiJoin(condition=[=($1, $2)], joinType=[semi])
+ HiveProject(d_date=[$2], d_week_seq=[$4])
+ HiveFilter(condition=[AND(IS NOT NULL($4), IS NOT NULL($2))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+ HiveProject(d_week_seq=[$4])
+ HiveFilter(condition=[AND(IN($2, 1998-01-02:DATE, 1998-10-15:DATE, 1998-11-10:DATE), IS NOT NULL($4))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
+CTE Suggestion:
+HiveProject(d_date_sk=[$0], d_date=[$2])
+ HiveFilter(condition=[IS NOT NULL($2)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveSortLimit(sort0=[$0], sort1=[$1], dir0=[ASC], dir1=[ASC], fetch=[100])
HiveProject(item_id=[$0], sr_item_qty=[$3], sr_dev=[*(/(/(CAST($3):DOUBLE, CAST(+(+($3, $1), $5)):DOUBLE), 3), 100)], cr_item_qty=[$1], cr_dev=[*(/(/(CAST($1):DOUBLE, CAST(+(+($3, $1), $5)):DOUBLE), 3), 100)], wr_item_qty=[$5], wr_dev=[*(/(/($6, CAST(+(+($3, $1), $5)):DOUBLE), 3), 100)], average=[/(CAST(+(+($3, $1), $5)):DECIMAL(19, 0), 3:DECIMAL(1, 0))])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query87.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query87.q.out
index 1b8898d6631..e95b750c38f 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query87.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query87.q.out
@@ -1,3 +1,8 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0], d_date=[$2])
+ HiveFilter(condition=[BETWEEN(false, $3, 1212, 1223)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveAggregate(group=[{}], agg#0=[count()])
HiveProject($f0=[$0], $f1=[$1], $f2=[$2], $f3=[$3], $f4=[$4])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query88.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query88.q.out
index e01c8b51a7f..f537b60140c 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query88.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query88.q.out
@@ -1,3 +1,18 @@
+CTE Suggestion:
+HiveProject(hd_demo_sk=[$0])
+ HiveFilter(condition=[AND(IN($3, 3, 0, 1), OR(<=($4, 5), <=($4, 2), <=($4, 3)), OR(AND(=($3, 3), <=($4, 5)), AND(=($3, 0), <=($4, 2)), AND(=($3, 1), <=($4, 3))))])
+ HiveTableScan(table=[[default, household_demographics]], table:alias=[household_demographics])
+
+CTE Suggestion:
+HiveProject(s_store_sk=[$0])
+ HiveFilter(condition=[=($5, _UTF-16LE'ese')])
+ HiveTableScan(table=[[default, store]], table:alias=[store])
+
+CTE Suggestion:
+HiveProject(ss_sold_time_sk=[$0], ss_hdemo_sk=[$4], ss_store_sk=[$6])
+ HiveFilter(condition=[AND(IS NOT NULL($4), IS NOT NULL($0), IS NOT NULL($6))])
+ HiveTableScan(table=[[default, store_sales]], table:alias=[store_sales])
+
Warning: Map Join MAPJOIN[605][bigTable=?] in task 'Reducer 5' is a cross product
Warning: Map Join MAPJOIN[604][bigTable=?] in task 'Reducer 5' is a cross product
Warning: Map Join MAPJOIN[603][bigTable=?] in task 'Reducer 5' is a cross product
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query90.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query90.q.out
index be1eba3555b..d5b6ac7baab 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query90.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query90.q.out
@@ -1,3 +1,17 @@
+CTE Suggestion:
+HiveJoin(condition=[=($2, $3)], joinType=[inner], algorithm=[none], cost=[not available])
+ HiveProject(ws_sold_time_sk=[$0], ws_ship_hdemo_sk=[$9], ws_web_page_sk=[$11])
+ HiveFilter(condition=[AND(IS NOT NULL($9), IS NOT NULL($0), IS NOT NULL($11))])
+ HiveTableScan(table=[[default, web_sales]], table:alias=[web_sales])
+ HiveProject(wp_web_page_sk=[$0])
+ HiveFilter(condition=[BETWEEN(false, $10, 5000, 5200)])
+ HiveTableScan(table=[[default, web_page]], table:alias=[web_page])
+
+CTE Suggestion:
+HiveProject(hd_demo_sk=[$0])
+ HiveFilter(condition=[=($3, 8)])
+ HiveTableScan(table=[[default, household_demographics]], table:alias=[household_demographics])
+
Warning: Map Join MAPJOIN[149][bigTable=?] in task 'Reducer 2' is a cross product
CBO PLAN:
HiveProject(am_pm_ratio=[/(CAST($0):DECIMAL(15, 4), CAST($1):DECIMAL(15, 4))])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query92.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query92.q.out
index b98fec5e068..afd7f4a5180 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query92.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query92.q.out
@@ -1,3 +1,8 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[BETWEEN(false, CAST($2):TIMESTAMP(9), 1998-03-18 00:00:00:TIMESTAMP(9), 1998-06-16 00:00:00:TIMESTAMP(9))])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveAggregate(group=[{}], agg#0=[sum($2)])
HiveJoin(condition=[=($0, $3)], joinType=[inner], algorithm=[none], cost=[not available])
diff --git a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query97.q.out b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query97.q.out
index 4ee10d53e78..5093b297c69 100644
--- a/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query97.q.out
+++ b/ql/src/test/results/clientpositive/perf/tpcds30tb/tez/cbo_query97.q.out
@@ -1,3 +1,8 @@
+CTE Suggestion:
+HiveProject(d_date_sk=[$0])
+ HiveFilter(condition=[BETWEEN(false, $3, 1212, 1223)])
+ HiveTableScan(table=[[default, date_dim]], table:alias=[date_dim])
+
CBO PLAN:
HiveAggregate(group=[{}], agg#0=[sum($0)], agg#1=[sum($1)], agg#2=[sum($2)])
HiveProject($f0=[CAST(CASE(AND(IS NULL($2), IS NOT NULL($0)), 1, 0)):INTEGER], $f1=[CAST(CASE(AND(IS NULL($0), IS NOT NULL($2)), 1, 0)):INTEGER], $f2=[CAST(CASE(AND(IS NOT NULL($0), IS NOT NULL($2)), 1, 0)):INTEGER])