You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by st...@apache.org on 2021/02/20 03:03:45 UTC
[impala] 01/08: IMPALA-10325: Parquet scan should use min/max
statistics to skip pages based on equi-join predicate
This is an automated email from the ASF dual-hosted git repository.
stigahuang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit ebb2e0663988d4fb597aeecb4defebbda1669f12
Author: Qifan Chen <qc...@cloudera.com>
AuthorDate: Tue Feb 16 13:37:21 2021 -0500
IMPALA-10325: Parquet scan should use min/max statistics to skip pages based on equi-join predicate
This patch adds a new class of predicates called overlap predicates
to aid in the acceptance or rejection of a row group, a page, or a
row in a Parquet table, utilizing the minimal and the maximal values
gathered from an equi hash join and the Parquet column index stats.
When a row group or page is rejected, all contained rows within are
rejected all together.
For example in the following query, the min and max in the overlap
predicate are computed from the join column from table 'b', and
are compared against the min/max of each row group or page at the
scan node for 'a'.
select straight_join count(*)
from lineitem a join [SHUFFLE] lineitem b
where a.l_shipdate = b.l_receiptdate
and b.l_commitdate = "1992-01-31";
An overlap predicate associated with the column type B in hash
table and scan column type A will be formed when both A and B are
of or can be converted to as:
1. booleans;
2. integers (tinyint, smallint, int, or bigint);
3. approximate numeric (float or double);
4. decimals with the same precision and scale;
5. strings;
6. date; or
7. timestamps.
The overlap predicate is implemented as a min/max filter and can be
observed in the explain output of a query.
A new query option 'minmax_filter_threshold' is provided to control
the new feature. Setting it to 0.0 disables the feature. Setting it
to a value > 0.0 but less than 1.0 provides a threshold. An overlap
predicate will be evaluated against a row group and possibly the
containing pages/rows, as long as its overlap ratio is less than the
threshold. The overlap ratio is the common area of the row group
and the filter, divided by the area of the row group.
A second query option, minmax_filtering_level, is provided to
specify the filtering scope:
1. ROW_GROUP: the overlap is only tested for row groups;
2. PAGE: the overlap is tested for both row groups and pages;
3. ROW: the overlap is for row groups, pages and rows.
Two new run-time profile counters are added to report the number of
row groups or pages filtered out via the overlap predicates
respectively:
1. NumRuntimeFilteredRowGroups
2. NumRuntimeFilteredPages
Two new column "Min value" and "Max value" are added to the
"Filter routing table" and "Final filter table" in profile to
display the min and the max values for a min/max filter.
Testing:
1. Unit tested on various column types with TPCH and TPCDS tables.
Benefits were significant when the join column on the outer table
is sorted and there exist many row groups or pages no overlapping
with the min/max filters;
2. Added following new tests:
a) In overlap_min_max_filters.test to demonstrate the number of
filtered out pages and row groups with the two new profile
counters;
b) In runtime-filter-propagation.test to demonstrate that the
overlap predicates work with different column types;
3. Core testing;
4. Performance measurement: the overal improvement with 3TB scale
TPCDS is at 1.45% with the filter threshold at 0.5 and filtering
level at ROW_GROUP. Good improvement (over 10%) are seen with
query 16, 25, 62, 83, 94 and 99, due to the join column
ship_date_sk being strongly correlated to the partition column
sold_date_sk.
To do in follow-up JIRAs:
1. Improve filtering efficiency;
2. Apply the overlap predicate on partition columns;
3. IR code-gen for various MinMaxFilter::EvalOverlap methods.
4. Address the current limitation that the "Min value" and
"Max value" columns may be empty for LOCAL filters.
Change-Id: I379405ee75b14929df7d6b5d20dabc6f51375691
Reviewed-on: http://gerrit.cloudera.org:8080/16720
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
be/src/exec/exec-node.h | 1 +
be/src/exec/hdfs-scan-node-base.cc | 3 +
be/src/exec/hdfs-scan-node-base.h | 10 +-
be/src/exec/hdfs-scanner-ir.cc | 2 +-
be/src/exec/hdfs-scanner.cc | 13 +-
be/src/exec/hdfs-scanner.h | 31 +-
be/src/exec/parquet/hdfs-parquet-scanner.cc | 429 ++++++++-
be/src/exec/parquet/hdfs-parquet-scanner.h | 73 ++
be/src/exec/parquet/parquet-column-stats.cc | 16 +
be/src/exec/parquet/parquet-column-stats.h | 11 +
be/src/exec/partitioned-hash-join-builder.cc | 6 +
be/src/exec/scan-node.cc | 2 +-
be/src/runtime/coordinator.cc | 38 +-
be/src/runtime/date-value.cc | 18 +
be/src/runtime/date-value.h | 7 +
be/src/runtime/raw-value.h | 8 +
be/src/runtime/runtime-filter-ir.cc | 22 +-
be/src/runtime/string-value-test.cc | 12 +
be/src/runtime/string-value.cc | 12 +
be/src/runtime/string-value.h | 4 +
be/src/runtime/timestamp-value.cc | 30 +
be/src/runtime/timestamp-value.h | 11 +-
be/src/service/query-options.cc | 20 +
be/src/service/query-options.h | 6 +-
be/src/util/debug-util.cc | 1 +
be/src/util/debug-util.h | 1 +
be/src/util/min-max-filter-ir.cc | 12 +-
be/src/util/min-max-filter-test.cc | 235 ++++-
be/src/util/min-max-filter.cc | 307 +++++-
be/src/util/min-max-filter.h | 174 ++--
common/thrift/ImpalaInternalService.thrift | 7 +
common/thrift/ImpalaService.thrift | 13 +
common/thrift/PlanNodes.thrift | 19 +
.../apache/impala/analysis/BinaryPredicate.java | 1 +
.../java/org/apache/impala/analysis/Predicate.java | 5 +
.../apache/impala/analysis/TupleDescriptor.java | 4 +
.../org/apache/impala/planner/HdfsScanNode.java | 179 ++++
.../impala/planner/RuntimeFilterGenerator.java | 79 +-
.../org/apache/impala/planner/PlannerTest.java | 26 +-
.../org/apache/impala/planner/PlannerTestBase.java | 2 +-
.../apache/impala/planner/TpcdsPlannerTest.java | 212 ++--
.../disable-runtime-overlap-filter.test | 29 +
.../queries/PlannerTest/setoperation-rewrite.test | 16 +-
.../queries/PlannerTest/tpcds/tpcds-q01.test | 54 +-
.../queries/PlannerTest/tpcds/tpcds-q02.test | 72 +-
.../queries/PlannerTest/tpcds/tpcds-q03.test | 24 +-
.../queries/PlannerTest/tpcds/tpcds-q04.test | 160 ++--
.../queries/PlannerTest/tpcds/tpcds-q05.test | 532 ++++++++++-
.../queries/PlannerTest/tpcds/tpcds-q06.test | 66 +-
.../queries/PlannerTest/tpcds/tpcds-q07.test | 36 +-
.../queries/PlannerTest/tpcds/tpcds-q08.test | 36 +-
.../queries/PlannerTest/tpcds/tpcds-q09.test | 180 ++--
.../queries/PlannerTest/tpcds/tpcds-q10a.test | 78 +-
.../queries/PlannerTest/tpcds/tpcds-q11.test | 115 +--
.../queries/PlannerTest/tpcds/tpcds-q12.test | 18 +-
.../queries/PlannerTest/tpcds/tpcds-q13.test | 429 ++++++++-
.../queries/PlannerTest/tpcds/tpcds-q14a.test | 282 ++++--
.../queries/PlannerTest/tpcds/tpcds-q14b.test | 228 +++--
.../queries/PlannerTest/tpcds/tpcds-q15.test | 30 +-
.../queries/PlannerTest/tpcds/tpcds-q16.test | 37 +-
.../queries/PlannerTest/tpcds/tpcds-q17.test | 69 +-
.../queries/PlannerTest/tpcds/tpcds-q18.test | 351 ++++++-
.../queries/PlannerTest/tpcds/tpcds-q19.test | 50 +-
.../queries/PlannerTest/tpcds/tpcds-q20.test | 18 +-
.../queries/PlannerTest/tpcds/tpcds-q21.test | 24 +-
.../queries/PlannerTest/tpcds/tpcds-q22.test | 207 +++-
.../queries/PlannerTest/tpcds/tpcds-q23a.test | 1009 ++++++++++++++++++--
.../queries/PlannerTest/tpcds/tpcds-q23b.test | 195 ++--
.../queries/PlannerTest/tpcds/tpcds-q24a.test | 114 ++-
.../queries/PlannerTest/tpcds/tpcds-q24b.test | 114 ++-
.../queries/PlannerTest/tpcds/tpcds-q25.test | 69 +-
.../queries/PlannerTest/tpcds/tpcds-q26.test | 31 +-
.../queries/PlannerTest/tpcds/tpcds-q27.test | 243 ++++-
.../queries/PlannerTest/tpcds/tpcds-q28.test | 73 +-
.../queries/PlannerTest/tpcds/tpcds-q29.test | 69 +-
.../queries/PlannerTest/tpcds/tpcds-q30.test | 73 +-
.../queries/PlannerTest/tpcds/tpcds-q31.test | 151 +--
.../queries/PlannerTest/tpcds/tpcds-q32.test | 39 +-
.../queries/PlannerTest/tpcds/tpcds-q33.test | 658 ++++++++++++-
.../queries/PlannerTest/tpcds/tpcds-q34.test | 38 +-
.../queries/PlannerTest/tpcds/tpcds-q35a.test | 78 +-
.../queries/PlannerTest/tpcds/tpcds-q36.test | 225 ++++-
.../queries/PlannerTest/tpcds/tpcds-q37.test | 31 +-
.../queries/PlannerTest/tpcds/tpcds-q38.test | 428 ++++++++-
.../queries/PlannerTest/tpcds/tpcds-q39a.test | 66 +-
.../queries/PlannerTest/tpcds/tpcds-q39b.test | 67 +-
.../queries/PlannerTest/tpcds/tpcds-q40.test | 31 +-
.../queries/PlannerTest/tpcds/tpcds-q41.test | 1 -
.../queries/PlannerTest/tpcds/tpcds-q42.test | 24 +-
.../queries/PlannerTest/tpcds/tpcds-q43.test | 24 +-
.../queries/PlannerTest/tpcds/tpcds-q44.test | 56 +-
.../queries/PlannerTest/tpcds/tpcds-q45.test | 279 +++++-
.../queries/PlannerTest/tpcds/tpcds-q46.test | 54 +-
.../queries/PlannerTest/tpcds/tpcds-q47.test | 90 +-
.../queries/PlannerTest/tpcds/tpcds-q48.test | 38 +-
.../queries/PlannerTest/tpcds/tpcds-q49.test | 36 +-
.../queries/PlannerTest/tpcds/tpcds-q50.test | 42 +-
.../queries/PlannerTest/tpcds/tpcds-q51.test | 24 +-
.../queries/PlannerTest/tpcds/tpcds-q52.test | 24 +-
.../queries/PlannerTest/tpcds/tpcds-q53.test | 30 +-
.../queries/PlannerTest/tpcds/tpcds-q54.test | 78 +-
.../queries/PlannerTest/tpcds/tpcds-q55.test | 24 +-
.../queries/PlannerTest/tpcds/tpcds-q56.test | 108 ++-
.../queries/PlannerTest/tpcds/tpcds-q57.test | 72 +-
.../queries/PlannerTest/tpcds/tpcds-q58.test | 138 +--
.../queries/PlannerTest/tpcds/tpcds-q59.test | 90 +-
.../queries/PlannerTest/tpcds/tpcds-q60.test | 668 ++++++++++++-
.../queries/PlannerTest/tpcds/tpcds-q61.test | 99 +-
.../queries/PlannerTest/tpcds/tpcds-q62.test | 30 +-
.../queries/PlannerTest/tpcds/tpcds-q63.test | 30 +-
.../queries/PlannerTest/tpcds/tpcds-q64.test | 221 ++++-
.../queries/PlannerTest/tpcds/tpcds-q65.test | 54 +-
.../queries/PlannerTest/tpcds/tpcds-q66.test | 99 +-
.../queries/PlannerTest/tpcds/tpcds-q67.test | 273 +++++-
.../queries/PlannerTest/tpcds/tpcds-q68.test | 54 +-
.../queries/PlannerTest/tpcds/tpcds-q69.test | 72 +-
.../queries/PlannerTest/tpcds/tpcds-q70.test | 360 ++++++-
.../queries/PlannerTest/tpcds/tpcds-q71.test | 54 +-
.../queries/PlannerTest/tpcds/tpcds-q72.test | 70 +-
.../queries/PlannerTest/tpcds/tpcds-q73.test | 38 +-
.../queries/PlannerTest/tpcds/tpcds-q74.test | 118 +--
.../queries/PlannerTest/tpcds/tpcds-q75.test | 138 +--
.../queries/PlannerTest/tpcds/tpcds-q76.test | 68 +-
.../queries/PlannerTest/tpcds/tpcds-q77.test | 732 +++++++++++++-
.../queries/PlannerTest/tpcds/tpcds-q78.test | 61 +-
.../queries/PlannerTest/tpcds/tpcds-q79.test | 38 +-
.../queries/PlannerTest/tpcds/tpcds-q80.test | 767 ++++++++++++++-
.../queries/PlannerTest/tpcds/tpcds-q81.test | 72 +-
.../queries/PlannerTest/tpcds/tpcds-q82.test | 42 +-
.../queries/PlannerTest/tpcds/tpcds-q83.test | 129 +--
.../queries/PlannerTest/tpcds/tpcds-q84.test | 42 +-
.../queries/PlannerTest/tpcds/tpcds-q85.test | 68 +-
.../queries/PlannerTest/tpcds/tpcds-q86.test | 180 +++-
.../queries/PlannerTest/tpcds/tpcds-q87.test | 410 +++++++-
.../queries/PlannerTest/tpcds/tpcds-q88.test | 246 +++--
.../queries/PlannerTest/tpcds/tpcds-q89.test | 30 +-
.../queries/PlannerTest/tpcds/tpcds-q90.test | 48 +-
.../queries/PlannerTest/tpcds/tpcds-q91.test | 54 +-
.../queries/PlannerTest/tpcds/tpcds-q92.test | 38 +-
.../queries/PlannerTest/tpcds/tpcds-q93.test | 24 +-
.../queries/PlannerTest/tpcds/tpcds-q94.test | 36 +-
.../queries/PlannerTest/tpcds/tpcds-q95.test | 416 +++++++-
.../queries/PlannerTest/tpcds/tpcds-q96.test | 36 +-
.../queries/PlannerTest/tpcds/tpcds-q97.test | 24 +-
.../queries/PlannerTest/tpcds/tpcds-q98.test | 24 +-
.../queries/PlannerTest/tpcds/tpcds-q99.test | 30 +-
.../queries/QueryTest/bloom_filters.test | 6 +-
.../queries/QueryTest/min_max_filters.test | 2 +-
.../queries/QueryTest/overlap_min_max_filters.test | 196 ++++
tests/query_test/test_runtime_filters.py | 22 +
150 files changed, 13045 insertions(+), 3411 deletions(-)
diff --git a/be/src/exec/exec-node.h b/be/src/exec/exec-node.h
index 0976907..97bbc60 100644
--- a/be/src/exec/exec-node.h
+++ b/be/src/exec/exec-node.h
@@ -264,6 +264,7 @@ class ExecNode {
int id() const { return id_; }
TPlanNodeType::type type() const { return type_; }
+ const PlanNode& plan_node() { return plan_node_; }
/// Returns a unique label for this ExecNode of the form "PLAN_NODE_TYPE(id=[int])",
/// for example, EXCHANGE_NODE (id=2).
diff --git a/be/src/exec/hdfs-scan-node-base.cc b/be/src/exec/hdfs-scan-node-base.cc
index 9efad6c..acdf679 100644
--- a/be/src/exec/hdfs-scan-node-base.cc
+++ b/be/src/exec/hdfs-scan-node-base.cc
@@ -224,6 +224,9 @@ Status HdfsScanPlanNode::Init(const TPlanNode& tnode, FragmentState* state) {
*min_max_row_desc, state, &min_max_conjuncts_));
}
+ // Transfer overlap predicate descs.
+ overlap_predicate_descs_ = tnode.hdfs_scan_node.overlap_predicate_descs;
+
RETURN_IF_ERROR(ProcessScanRangesAndInitSharedState(state));
state->CheckAndAddCodegenDisabledMessage(codegen_status_msgs_);
diff --git a/be/src/exec/hdfs-scan-node-base.h b/be/src/exec/hdfs-scan-node-base.h
index 3591a94..4036bd9 100644
--- a/be/src/exec/hdfs-scan-node-base.h
+++ b/be/src/exec/hdfs-scan-node-base.h
@@ -279,7 +279,15 @@ class HdfsScanPlanNode : public ScanPlanNode {
/// Conjuncts to evaluate on parquet::Statistics.
std::vector<ScalarExpr*> min_max_conjuncts_;
- /// Tuple id resolved in Init() to set tuple_desc_ .
+ /// The list of overlap predicate descs. Each is used to find out whether the range
+ /// of values of a data item overlap with a min/max filter. Data structure wise, each
+ /// desc is composed of the ID of the min/max filter, the slot index in
+ /// 'min_max_tuple_desc_' to hold the min value of the data item and the actual overlap
+ /// predicate. The next slot after the slot index implicitly holds the max value of
+ /// the data item.
+ std::vector<TOverlapPredicateDesc> overlap_predicate_descs_;
+
+ /// Tuple id resolved in Init() to set tuple_desc_.
int tuple_id_ = -1;
/// Descriptor for tuples this scan node constructs
diff --git a/be/src/exec/hdfs-scanner-ir.cc b/be/src/exec/hdfs-scanner-ir.cc
index 1008ccf..c3b557b 100644
--- a/be/src/exec/hdfs-scanner-ir.cc
+++ b/be/src/exec/hdfs-scanner-ir.cc
@@ -101,7 +101,7 @@ bool HdfsScanner::EvalRuntimeFilter(int i, TupleRow* row) {
LocalFilterStats* stats = &filter_stats_[i];
const FilterContext* ctx = filter_ctxs_[i];
++stats->total_possible;
- if (stats->enabled && ctx->filter->HasFilter()) {
+ if (stats->enabled_for_row && ctx->filter->HasFilter()) {
++stats->considered;
if (!ctx->Eval(row)) {
++stats->rejected;
diff --git a/be/src/exec/hdfs-scanner.cc b/be/src/exec/hdfs-scanner.cc
index ec9f644..6275aa2 100644
--- a/be/src/exec/hdfs-scanner.cc
+++ b/be/src/exec/hdfs-scanner.cc
@@ -644,7 +644,7 @@ Status HdfsScanner::CodegenInitTuple(
// }
//
// EvalRuntimeFilter() is the same as the cross-compiled version except EvalOneFilter()
-// is replaced with the one generated by CodegenEvalOneFilter().
+// is replaced with the one generated by FilterContext::CodegenEval().
Status HdfsScanner::CodegenEvalRuntimeFilters(
LlvmCodeGen* codegen, const vector<ScalarExpr*>& filter_exprs, llvm::Function** fn) {
llvm::LLVMContext& context = codegen->context();
@@ -673,6 +673,7 @@ Status HdfsScanner::CodegenEvalRuntimeFilters(
DCHECK_GT(num_filters, 0);
for (int i = 0; i < num_filters; ++i) {
+ // Make a clone of HdfsScanner::EvalRuntimeFilter(int i, TupleRow* row)
llvm::Function* eval_runtime_filter_fn =
codegen->GetFunction(IRFunction::HDFS_SCANNER_EVAL_RUNTIME_FILTER, true);
DCHECK(eval_runtime_filter_fn != nullptr);
@@ -685,17 +686,25 @@ Status HdfsScanner::CodegenEvalRuntimeFilters(
&eval_one_filter_fn));
DCHECK(eval_one_filter_fn != nullptr);
+ // "FilterContext4Eval" is FilterContext::Eval(impala::TupleRow*).
+ // Replace ctx->Eval(row) in the above cloned function with its LLVM version
+ // which is 'eval_one_filter_fn'.
int replaced = codegen->ReplaceCallSites(eval_runtime_filter_fn, eval_one_filter_fn,
"FilterContext4Eval");
DCHECK_REPLACE_COUNT(replaced, 1);
+ // Create a call to the above LLVM optimized eval_runtime_filter_fn.
llvm::Value* idx = codegen->GetI32Constant(i);
llvm::Value* passed_filter = builder.CreateCall(
eval_runtime_filter_fn, llvm::ArrayRef<llvm::Value*>({this_arg, idx, row_arg}));
+ // Create a new block "continue" at the end of eval_runtime_filters_fn block.
llvm::BasicBlock* continue_block =
llvm::BasicBlock::Create(context, "continue", eval_runtime_filters_fn);
+ // Create a condition on top of eval_runtime_filter_fn.
builder.CreateCondBr(passed_filter, continue_block, row_rejected_block);
+ // Mark 'continue_block' the location to accumulate any additional LLVM code
+ // for the next filter (if any).
builder.SetInsertPoint(continue_block);
}
builder.CreateRet(codegen->true_value());
@@ -809,7 +818,7 @@ void HdfsScanner::CheckFiltersEffectiveness() {
double reject_ratio = stats->rejected / static_cast<double>(stats->considered);
if (filter->AlwaysTrue() ||
reject_ratio < FLAGS_min_filter_reject_ratio) {
- stats->enabled = 0;
+ stats->enabled_for_row = 0;
}
}
}
diff --git a/be/src/exec/hdfs-scanner.h b/be/src/exec/hdfs-scanner.h
index 5a92abc..8e35124 100644
--- a/be/src/exec/hdfs-scanner.h
+++ b/be/src/exec/hdfs-scanner.h
@@ -181,6 +181,18 @@ class HdfsScanner {
return desc.byte_size();
}
+ /// Find the index of a filter in filter_ctxs_ (a vector) through id.
+ int FindFilterIndex(int filter_id) {
+ for (int i = 0; i < filter_ctxs_.size(); ++i) {
+ const RuntimeFilter* filter = filter_ctxs_[i]->filter;
+ if (filter && filter->id() == filter_id) {
+ return i;
+ }
+ }
+ DCHECK(false);
+ return -1;
+ }
+
/// Scanner subclasses must implement these static functions as well. Unfortunately,
/// c++ does not allow static virtual functions.
@@ -325,13 +337,22 @@ class HdfsScanner {
int64_t total_possible;
/// Use known-width type to act as logical boolean. Set to 1 if corresponding filter
- /// in filter_ctxs_ should be applied, 0 if it was ineffective and was disabled.
- uint8_t enabled;
+ /// in filter_ctxs_ should be applied at row level, 0 if it was ineffective and was
+ /// disabled.
+ uint8_t enabled_for_row;
- /// Padding to ensure structs do not straddle cache-line boundary.
- uint8_t padding[7];
+ /// Apply the filter at page level only.
+ uint8_t enabled_for_page;
- LocalFilterStats() : considered(0), rejected(0), total_possible(0), enabled(1) { }
+ /// Padding to ensure structs do not straddle cache-line boundary.
+ uint8_t padding[6];
+
+ LocalFilterStats()
+ : considered(0),
+ rejected(0),
+ total_possible(0),
+ enabled_for_row(1),
+ enabled_for_page(1) {}
};
/// Cached runtime filter contexts, one for each filter that applies to this column.
diff --git a/be/src/exec/parquet/hdfs-parquet-scanner.cc b/be/src/exec/parquet/hdfs-parquet-scanner.cc
index d71898e..3fc7c79 100644
--- a/be/src/exec/parquet/hdfs-parquet-scanner.cc
+++ b/be/src/exec/parquet/hdfs-parquet-scanner.cc
@@ -93,7 +93,9 @@ HdfsParquetScanner::HdfsParquetScanner(HdfsScanNodeBase* scan_node, RuntimeState
process_footer_timer_stats_(nullptr),
num_cols_counter_(nullptr),
num_stats_filtered_row_groups_counter_(nullptr),
+ num_minmax_filtered_row_groups_counter_(nullptr),
num_row_groups_counter_(nullptr),
+ num_minmax_filtered_pages_counter_(nullptr),
num_scanners_with_no_reads_counter_(nullptr),
num_dict_filtered_row_groups_counter_(nullptr),
parquet_compressed_page_size_counter_(nullptr),
@@ -111,6 +113,9 @@ Status HdfsParquetScanner::Open(ScannerContext* context) {
num_stats_filtered_row_groups_counter_ =
ADD_COUNTER(scan_node_->runtime_profile(), "NumStatsFilteredRowGroups",
TUnit::UNIT);
+ num_minmax_filtered_row_groups_counter_ =
+ ADD_COUNTER(scan_node_->runtime_profile(), "NumRuntimeFilteredRowGroups",
+ TUnit::UNIT);
num_row_groups_counter_ =
ADD_COUNTER(scan_node_->runtime_profile(), "NumRowGroups", TUnit::UNIT);
num_row_groups_with_page_index_counter_ =
@@ -118,6 +123,8 @@ Status HdfsParquetScanner::Open(ScannerContext* context) {
TUnit::UNIT);
num_stats_filtered_pages_counter_ =
ADD_COUNTER(scan_node_->runtime_profile(), "NumStatsFilteredPages", TUnit::UNIT);
+ num_minmax_filtered_pages_counter_ =
+ ADD_COUNTER(scan_node_->runtime_profile(), "NumRuntimeFilteredPages", TUnit::UNIT);
num_pages_counter_ =
ADD_COUNTER(scan_node_->runtime_profile(), "NumPages", TUnit::UNIT);
num_scanners_with_no_reads_counter_ =
@@ -466,6 +473,52 @@ Status HdfsParquetScanner::GetNextInternal(RowBatch* row_batch) {
return Status::OK();
}
+Status HdfsParquetScanner::ResolveSchemaForStatFiltering(SlotDescriptor* slot_desc,
+ bool* missing_field, SchemaNode** schema_node_ptr) {
+ DCHECK(missing_field);
+ *missing_field = false;
+ // Resolve column path.
+ SchemaNode* node = nullptr;
+ bool pos_field;
+ RETURN_IF_ERROR(schema_resolver_->ResolvePath(
+ slot_desc->col_path(), &node, &pos_field, missing_field));
+
+ if (pos_field) {
+ // The planner should not send predicates with 'pos' for stats filtering to the BE.
+ // In case there is a bug, we return an error, which will abort the query.
+ stringstream err;
+ err << "Statistics not supported for pos fields: " << slot_desc->DebugString();
+ DCHECK(false) << err.str();
+ return Status(err.str());
+ }
+
+ if (schema_node_ptr) *schema_node_ptr = node;
+
+ return Status::OK();
+}
+
+ColumnStatsReader HdfsParquetScanner::CreateStatsReader(
+ const parquet::FileMetaData& file_metadata, const parquet::RowGroup& row_group,
+ SchemaNode* node, const ColumnType& col_type) {
+ DCHECK(node);
+
+ int col_idx = node->col_idx;
+ DCHECK_LT(col_idx, row_group.columns.size());
+
+ const vector<parquet::ColumnOrder>& col_orders = file_metadata.column_orders;
+ const parquet::ColumnOrder* col_order =
+ col_idx < col_orders.size() ? &col_orders[col_idx] : nullptr;
+
+ const parquet::ColumnChunk& col_chunk = row_group.columns[col_idx];
+
+ DCHECK(node->element != nullptr);
+
+ ColumnStatsReader stat_reader =
+ CreateColumnStatsReader(col_chunk, col_type, col_order, *node->element);
+
+ return stat_reader;
+}
+
Status HdfsParquetScanner::EvaluateStatsConjuncts(
const parquet::FileMetaData& file_metadata, const parquet::RowGroup& row_group,
bool* skip_row_group) {
@@ -481,17 +534,16 @@ Status HdfsParquetScanner::EvaluateStatsConjuncts(
DCHECK(min_max_tuple_ != nullptr);
min_max_tuple_->Init(tuple_size);
- DCHECK_EQ(min_max_tuple_desc->slots().size(), min_max_conjunct_evals_.size());
+ DCHECK_GE(min_max_tuple_desc->slots().size(), min_max_conjunct_evals_.size());
for (int i = 0; i < min_max_conjunct_evals_.size(); ++i) {
+
SlotDescriptor* slot_desc = min_max_tuple_desc->slots()[i];
ScalarExprEvaluator* eval = min_max_conjunct_evals_[i];
- // Resolve column path to determine col idx.
+ bool missing_field = false;
SchemaNode* node = nullptr;
- bool pos_field;
- bool missing_field;
- RETURN_IF_ERROR(schema_resolver_->ResolvePath(slot_desc->col_path(),
- &node, &pos_field, &missing_field));
+
+ RETURN_IF_ERROR(ResolveSchemaForStatFiltering(slot_desc, &missing_field, &node));
if (missing_field) {
// We are selecting a column that is not in the file. We would set its slot to NULL
@@ -502,33 +554,11 @@ Status HdfsParquetScanner::EvaluateStatsConjuncts(
break;
}
- if (pos_field) {
- // The planner should not send predicates with 'pos' for stats filtering to the BE.
- // In case there is a bug, we return an error, which will abort the query.
- stringstream err;
- err << "Statistics not supported for pos fields: " << slot_desc->DebugString();
- DCHECK(false) << err.str();
- return Status(err.str());
- }
-
- int col_idx = node->col_idx;
- DCHECK_LT(col_idx, row_group.columns.size());
-
- const vector<parquet::ColumnOrder>& col_orders = file_metadata.column_orders;
- const parquet::ColumnOrder* col_order = col_idx < col_orders.size() ?
- &col_orders[col_idx] : nullptr;
-
- const parquet::ColumnChunk& col_chunk = row_group.columns[col_idx];
- const ColumnType& col_type = slot_desc->type();
-
- DCHECK(node->element != nullptr);
-
- ColumnStatsReader stat_reader = CreateColumnStatsReader(col_chunk, col_type,
- col_order, *node->element);
+ ColumnStatsReader stats_reader =
+ CreateStatsReader(file_metadata, row_group, node, slot_desc->type());
- int64_t null_count = 0;
- bool null_count_result = stat_reader.ReadNullCountStat(&null_count);
- if (null_count_result && null_count == col_chunk.meta_data.num_values) {
+ bool all_nulls = false;
+ if (stats_reader.AllNulls(&all_nulls) && all_nulls) {
*skip_row_group = true;
break;
}
@@ -538,7 +568,7 @@ Status HdfsParquetScanner::EvaluateStatsConjuncts(
if (!ColumnStatsReader::GetRequiredStatsField(fn_name, &stats_field)) continue;
void* slot = min_max_tuple_->GetSlot(slot_desc->tuple_offset());
- bool stats_read = stat_reader.ReadFromThrift(stats_field, slot);
+ bool stats_read = stats_reader.ReadFromThrift(stats_field, slot);
if (stats_read) {
TupleRow row;
@@ -556,6 +586,146 @@ Status HdfsParquetScanner::EvaluateStatsConjuncts(
return Status::OK();
}
+Status HdfsParquetScanner::EvaluateOverlapForRowGroup(
+ const parquet::FileMetaData& file_metadata, const parquet::RowGroup& row_group,
+ bool* skip_row_group) {
+ *skip_row_group = false;
+
+ if (!state_->query_options().parquet_read_statistics) return Status::OK();
+
+ const TupleDescriptor* min_max_tuple_desc = scan_node_->min_max_tuple_desc();
+ if (GetOverlapPredicateDescs().size() > 0 && !min_max_tuple_desc) {
+ stringstream err;
+ err << "min_max_tuple_desc is null.";
+ DCHECK(false) << err.str();
+ return Status(err.str());
+ }
+
+ DCHECK(min_max_tuple_ != nullptr);
+ min_max_tuple_->Init(min_max_tuple_desc->byte_size());
+
+ // The total number slots in min_max_tuple_ should be equal to or larger than
+ // the number of min/max conjuncts. The extra number slots are for the overlap
+ // predicates. # min_max_conjuncts + 2 * # overlap predicates = # min_max_slots.
+ DCHECK_GE(min_max_tuple_desc->slots().size(), min_max_conjunct_evals_.size());
+
+ TMinmaxFilteringLevel::type level = state_->query_options().minmax_filtering_level;
+ float threshold = (float)(state_->query_options().minmax_filter_threshold);
+
+ for (auto desc: GetOverlapPredicateDescs()) {
+
+ int filter_id = desc.filter_id;
+ int slot_idx = desc.slot_index;
+ MinMaxFilter* minmax_filter = FindMinMaxFilter(FindFilterIndex(filter_id));
+
+ if (!minmax_filter) continue;
+
+ SlotDescriptor* slot_desc = min_max_tuple_desc->slots()[slot_idx];
+
+ bool missing_field = false;
+ SchemaNode* node = nullptr;
+
+ RETURN_IF_ERROR(ResolveSchemaForStatFiltering(slot_desc, &missing_field, &node));
+
+ if (missing_field) {
+ // We are selecting a column that is not in the file. We would set its slot to NULL
+ // during the scan, so any predicate would evaluate to false. Return early. NULL
+ // comparisons cannot happen here, since predicates with NULL literals are filtered
+ // in the frontend.
+ *skip_row_group = true;
+ break;
+ }
+ ColumnStatsReader stats_reader =
+ CreateStatsReader(file_metadata, row_group, node, slot_desc->type());
+
+ bool all_nulls = false;
+ if (stats_reader.AllNulls(&all_nulls) && all_nulls) {
+ *skip_row_group = true;
+ break;
+ }
+
+ void* min_slot = nullptr;
+ void* max_slot = nullptr;
+ GetMinMaxSlotsForOverlapPred(slot_idx, &min_slot, &max_slot);
+
+ bool value_read = stats_reader.ReadMinMaxFromThrift(min_slot, max_slot);
+ if (!value_read) {
+ continue;
+ }
+
+ const ColumnType& col_type = slot_desc->type();
+
+ float overlap_ratio =
+ minmax_filter->ComputeOverlapRatio(col_type, min_slot, max_slot);
+
+ /// Compute the likelyhood of this filter being effective. Threshold 'threshold' is
+ /// the upper bound for the overlapping ratio. Any filter with an overlap ratio
+ /// (>0.0) less than 'threshold' will undertake overlap check at the page and
+ /// the row level when the filtering level control allows.
+ bool worthiness = !(minmax_filter->AlwaysTrue()) && (overlap_ratio < threshold);
+
+ VLOG(3) << "Try to filter out a rowgroup via overlap predicate:"
+ << " fid=" << filter_id
+ << ", SchemaNode=" << node->DebugString()
+ << ", columnType=" << col_type.DebugString()
+ << ", overlap ratio=" << overlap_ratio
+ << ", threshold=" << threshold
+ << ", worthiness=" << worthiness
+ << ", enabled for page=" <<
+ ((level != TMinmaxFilteringLevel::ROW_GROUP) && worthiness)
+ << ", enabled for row=" <<
+ ((level == TMinmaxFilteringLevel::ROW) && worthiness)
+ << ", data min="
+ << RawValue::PrintValue(min_slot, col_type, col_type.scale)
+ << ", data max="
+ << RawValue::PrintValue(max_slot, col_type, col_type.scale)
+ << ", content=" << minmax_filter->DebugString();
+
+ /// Find the index of the filter that is common in data structure
+ /// filter_ctxs_ and filter_stats_.
+ int idx = FindFilterIndex(filter_id);
+ DCHECK(idx >= 0);
+
+ /// If not overlapping with this particular filter, the row group can be filtered
+ /// out safely.
+ if (overlap_ratio == 0.0) {
+ VLOG(3) << "The rowgroup was filtered out.";
+ *skip_row_group = true;
+
+ /// Update the row groups stats: rejected.
+ filter_ctxs_[idx]->stats->IncrCounters(FilterStats::ROW_GROUPS_KEY, 1, 1, 1);
+ break;
+ }
+
+ /// Update two fields in the entry at index 'idx' in local filter stats so that
+ /// the info can be used when checking out the pages in the group in
+ /// FindSkipRangesForPagesWithMinMaxFilters() and rows in these pages in
+ /// HdfsScanner::EvalRuntimeFilter(). The worthiness flag is ANDed into the enabled
+ /// field so that the state in enabled will never go back to 1 once set to 0.
+ /// The filtering level control is considered too.
+ filter_stats_[idx].enabled_for_row &=
+ (level == TMinmaxFilteringLevel::ROW) && worthiness;
+ filter_stats_[idx].enabled_for_page &=
+ (level != TMinmaxFilteringLevel::ROW_GROUP) && worthiness;
+
+ /// Update the row groups stats: no rejection.
+ filter_ctxs_[idx]->stats->IncrCounters(FilterStats::ROW_GROUPS_KEY, 1, 1, 0);
+ }
+
+ return Status::OK();
+}
+
+bool HdfsParquetScanner::ShouldProcessPageIndex() {
+ if (!state_->query_options().parquet_read_page_index) return false;
+ if (!min_max_conjunct_evals_.empty()) return true;
+ for (auto desc : GetOverlapPredicateDescs()) {
+ if (IsFilterWorthyForOverlapCheck(FindFilterIndex(desc.filter_id))) {
+ return true;
+ }
+ }
+ return false;
+}
+
Status HdfsParquetScanner::NextRowGroup() {
const ScanRange* split_range = static_cast<ScanRangeMetadata*>(
metadata_range_->meta_data())->original_split;
@@ -618,7 +788,7 @@ Status HdfsParquetScanner::NextRowGroup() {
COUNTER_ADD(num_row_groups_with_page_index_counter_, 1);
}
- // Evaluate row group statistics.
+ // Evaluate row group statistics with stats conjuncts.
bool skip_row_group_on_stats;
RETURN_IF_ERROR(
EvaluateStatsConjuncts(file_metadata_, row_group, &skip_row_group_on_stats));
@@ -627,9 +797,17 @@ Status HdfsParquetScanner::NextRowGroup() {
continue;
}
- // Evaluate page index.
- if (!min_max_conjunct_evals_.empty() &&
- state_->query_options().parquet_read_page_index) {
+ // Evaluate row group statistics with min/max filters.
+ bool skip_row_group_on_minmax;
+ RETURN_IF_ERROR(
+ EvaluateOverlapForRowGroup(file_metadata_, row_group, &skip_row_group_on_minmax));
+ if (skip_row_group_on_minmax) {
+ COUNTER_ADD(num_minmax_filtered_row_groups_counter_, 1);
+ continue;
+ }
+
+ // Evaluate page index with min-max conjuncts and/or min/max overlap predicates.
+ if (ShouldProcessPageIndex()) {
Status page_index_status = ProcessPageIndex();
if (!page_index_status.ok()) {
RETURN_IF_ERROR(state_->LogOrReturnError(page_index_status.msg()));
@@ -711,6 +889,19 @@ bool HdfsParquetScanner::ReadStatFromIndex(const ColumnStatsReader& stats_reader
return false;
}
+bool HdfsParquetScanner::ReadStatFromIndex(const ColumnStatsReader& stats_reader,
+ const parquet::ColumnIndex& column_index, int page_idx, bool* is_null_page,
+ void* min_slot, void* max_slot) {
+ *is_null_page = column_index.null_pages[page_idx];
+ if (*is_null_page) return false;
+ return (
+ stats_reader.ReadFromString(ColumnStatsReader::StatsField::MIN,
+ column_index.min_values[page_idx], &column_index.max_values[page_idx], min_slot)
+ && stats_reader.ReadFromString(ColumnStatsReader::StatsField::MAX,
+ column_index.max_values[page_idx], &column_index.min_values[page_idx],
+ max_slot));
+}
+
void HdfsParquetScanner::ResetPageFiltering() {
filter_pages_ = false;
candidate_ranges_.clear();
@@ -732,6 +923,134 @@ Status HdfsParquetScanner::ProcessPageIndex() {
return Status::OK();
}
+const vector<TOverlapPredicateDesc>& HdfsParquetScanner::GetOverlapPredicateDescs() {
+ const HdfsScanPlanNode& pnode =
+ static_cast<const HdfsScanPlanNode&>(scan_node_->plan_node());
+ return pnode.overlap_predicate_descs_;
+}
+
+void HdfsParquetScanner::GetMinMaxSlotsForOverlapPred(
+ int overlap_slot_idx, void** min_slot, void** max_slot) {
+ DCHECK(min_slot);
+ DCHECK(max_slot);
+ SlotDescriptor* min_slot_desc =
+ scan_node_->min_max_tuple_desc()->slots()[overlap_slot_idx];
+ SlotDescriptor* max_slot_desc =
+ scan_node_->min_max_tuple_desc()->slots()[overlap_slot_idx + 1];
+ *min_slot = min_max_tuple_->GetSlot(min_slot_desc->tuple_offset());
+ *max_slot = min_max_tuple_->GetSlot(max_slot_desc->tuple_offset());
+}
+
+MinMaxFilter* HdfsParquetScanner::FindMinMaxFilter(int filter_idx) {
+ if (filter_idx >= 0 && filter_idx < filter_ctxs_.size()) {
+ const RuntimeFilter* filter = filter_ctxs_[filter_idx]->filter;
+ if (filter && filter->is_min_max_filter()) {
+ return filter->get_min_max();
+ }
+ }
+ return nullptr;
+}
+
+bool HdfsParquetScanner::IsFilterWorthyForOverlapCheck(int filter_idx) {
+ if (filter_idx >= 0 && filter_idx < filter_stats_.size()) {
+ LocalFilterStats* stats = &filter_stats_[filter_idx];
+ return stats && stats->enabled_for_page;
+ }
+ DCHECK(false);
+ return true;
+}
+
+Status HdfsParquetScanner::FindSkipRangesForPagesWithMinMaxFilters(
+ vector<RowRange>* skip_ranges) {
+ DCHECK(skip_ranges);
+ const TupleDescriptor* min_max_tuple_desc = scan_node_->min_max_tuple_desc();
+ if (GetOverlapPredicateDescs().size() > 0 && !min_max_tuple_desc) {
+ stringstream err;
+ err << "min_max_tuple_desc is null.";
+ DCHECK(false) << err.str();
+ return Status(err.str());
+ }
+
+ min_max_tuple_->Init(min_max_tuple_desc->byte_size());
+ parquet::RowGroup& row_group = file_metadata_.row_groups[row_group_idx_];
+
+ int filtered_pages = 0;
+
+ for (auto desc: GetOverlapPredicateDescs()) {
+ int filter_id = desc.filter_id;
+ int slot_idx = desc.slot_index;
+
+ int filter_idx = FindFilterIndex(filter_id);
+ MinMaxFilter* minmax_filter = FindMinMaxFilter(filter_idx);
+ if (!minmax_filter || !IsFilterWorthyForOverlapCheck(filter_idx)) {
+ continue;
+ }
+
+ // This is the desc for the min slot, from which we find the stats_reader
+ // through col_path.
+ SlotDescriptor* slot_desc = min_max_tuple_desc->slots()[slot_idx];
+
+ bool missing_field = false;
+ SchemaNode* node = nullptr;
+
+ RETURN_IF_ERROR(ResolveSchemaForStatFiltering(slot_desc, &missing_field, &node));
+
+ if (missing_field) {
+ continue;
+ }
+
+ int col_idx = node->col_idx;
+ ColumnStatsReader stats_reader =
+ CreateStatsReader(file_metadata_, row_group, node, slot_desc->type());
+
+ DCHECK_LT(col_idx, row_group.columns.size());
+ const parquet::ColumnChunk& col_chunk = row_group.columns[col_idx];
+ if (col_chunk.column_index_length == 0) continue;
+
+ parquet::ColumnIndex column_index;
+ RETURN_IF_ERROR(page_index_.DeserializeColumnIndex(col_chunk, &column_index));
+
+ const ColumnType& col_type = slot_desc->type();
+ void* min_slot = nullptr;
+ void* max_slot = nullptr;
+ GetMinMaxSlotsForOverlapPred(slot_idx, &min_slot, &max_slot);
+
+ const int num_of_pages = column_index.null_pages.size();
+
+ VLOG(3) << "Try to filter out pages via overlap predicate."
+ << " fid=" << filter_id << ", columnType=" << col_type.DebugString()
+ << ", data min="
+ << RawValue::PrintValue(min_slot, col_type, col_type.scale)
+ << ", data max="
+ << RawValue::PrintValue(max_slot, col_type, col_type.scale)
+ << ", filter=" << minmax_filter->DebugString();
+
+ for (int page_idx = 0; page_idx < num_of_pages; ++page_idx) {
+ bool is_null_page = false;
+ if (!ReadStatFromIndex(
+ stats_reader, column_index, page_idx, &is_null_page, min_slot, max_slot)) {
+ continue;
+ }
+
+ if (is_null_page || !minmax_filter->EvalOverlap(col_type, min_slot, max_slot)) {
+ VLOG(3) << "Page " << page_idx << " was filtered out.";
+ BaseScalarColumnReader* scalar_reader = scalar_reader_map_[col_idx];
+ RETURN_IF_ERROR(
+ page_index_.DeserializeOffsetIndex(col_chunk, &scalar_reader->offset_index_));
+ RowRange row_range;
+ GetRowRangeForPage(row_group, scalar_reader->offset_index_, page_idx, &row_range);
+ skip_ranges->push_back(row_range);
+ filtered_pages++;
+ }
+ }
+ }
+
+ if (filtered_pages > 0) {
+ COUNTER_ADD(num_minmax_filtered_pages_counter_, filtered_pages);
+ }
+ return Status::OK();
+}
+
Status HdfsParquetScanner::EvaluatePageIndex() {
parquet::RowGroup& row_group = file_metadata_.row_groups[row_group_idx_];
vector<RowRange> skip_ranges;
@@ -740,14 +1059,18 @@ Status HdfsParquetScanner::EvaluatePageIndex() {
ScalarExprEvaluator* eval = min_max_conjunct_evals_[i];
SlotDescriptor* slot_desc = scan_node_->min_max_tuple_desc()->slots()[i];
- // Resolve column path to determine col idx.
+ bool missing_field = false;
SchemaNode* node = nullptr;
- bool pos_field, missing_field;
- RETURN_IF_ERROR(schema_resolver_->ResolvePath(slot_desc->col_path(), &node,
- &pos_field, &missing_field));
- if (pos_field || missing_field) continue;
- int col_idx = node->col_idx;
+ RETURN_IF_ERROR(ResolveSchemaForStatFiltering(slot_desc, &missing_field, &node));
+
+ if (missing_field) {
+ continue;
+ }
+ int col_idx = node->col_idx;;
+ ColumnStatsReader stats_reader =
+ CreateStatsReader(file_metadata_, row_group, node, slot_desc->type());
+
DCHECK_LT(col_idx, row_group.columns.size());
const parquet::ColumnChunk& col_chunk = row_group.columns[col_idx];
if (col_chunk.column_index_length == 0) continue;
@@ -755,13 +1078,6 @@ Status HdfsParquetScanner::EvaluatePageIndex() {
parquet::ColumnIndex column_index;
RETURN_IF_ERROR(page_index_.DeserializeColumnIndex(col_chunk, &column_index));
- const ColumnType& col_type = slot_desc->type();
- const vector<parquet::ColumnOrder>& col_orders = file_metadata_.column_orders;
- const parquet::ColumnOrder* col_order = col_idx < col_orders.size() ?
- &col_orders[col_idx] : nullptr;
- ColumnStatsReader stats_reader = CreateColumnStatsReader(col_chunk, col_type,
- col_order, *node->element);
-
min_max_tuple_->Init(scan_node_->min_max_tuple_desc()->byte_size());
void* slot = min_max_tuple_->GetSlot(slot_desc->tuple_offset());
@@ -772,8 +1088,8 @@ Status HdfsParquetScanner::EvaluatePageIndex() {
for (int page_idx = 0; page_idx < num_of_pages; ++page_idx) {
bool value_read, is_null_page;
- value_read = ReadStatFromIndex(stats_reader, column_index, page_idx, stats_field,
- &is_null_page, slot);
+ value_read = ReadStatFromIndex(
+ stats_reader, column_index, page_idx, stats_field, &is_null_page, slot);
if (!is_null_page && !value_read) continue;
TupleRow row;
row.SetTuple(0, min_max_tuple_);
@@ -788,6 +1104,13 @@ Status HdfsParquetScanner::EvaluatePageIndex() {
}
}
}
+
+ // On top of min/max conjuncts, apply min/max filters to filter out pages.
+ if (state_->query_options().minmax_filtering_level
+ != TMinmaxFilteringLevel::ROW_GROUP) {
+ RETURN_IF_ERROR(FindSkipRangesForPagesWithMinMaxFilters(&skip_ranges));
+ }
+
if (skip_ranges.empty()) return Status::OK();
for (BaseScalarColumnReader* scalar_reader : scalar_readers_) {
@@ -1556,8 +1879,8 @@ Status HdfsParquetScanner::CreateColumnReaders(const TupleDescriptor& tuple_desc
Status HdfsParquetScanner::CreateCountingReader(const SchemaPath& parent_path,
const ParquetSchemaResolver& schema_resolver, ParquetColumnReader** reader) {
SchemaNode* parent_node;
- bool pos_field;
bool missing_field;
+ bool pos_field;
RETURN_IF_ERROR(schema_resolver.ResolvePath(
parent_path, &parent_node, &pos_field, &missing_field));
diff --git a/be/src/exec/parquet/hdfs-parquet-scanner.h b/be/src/exec/parquet/hdfs-parquet-scanner.h
index 2986f1f..9f5cd69 100644
--- a/be/src/exec/parquet/hdfs-parquet-scanner.h
+++ b/be/src/exec/parquet/hdfs-parquet-scanner.h
@@ -461,6 +461,10 @@ class HdfsParquetScanner : public HdfsColumnarScanner {
/// row group level statistics, or page level statistics.
RuntimeProfile::Counter* num_stats_filtered_row_groups_counter_;
+ /// Number of row groups that are skipped because of Parquet row group level statistics
+ /// and HJ min/max filters.
+ RuntimeProfile::Counter* num_minmax_filtered_row_groups_counter_;
+
/// Number of row groups that need to be read.
RuntimeProfile::Counter* num_row_groups_counter_;
@@ -470,6 +474,10 @@ class HdfsParquetScanner : public HdfsColumnarScanner {
/// Number of pages that are skipped because of Parquet page statistics.
RuntimeProfile::Counter* num_stats_filtered_pages_counter_;
+ /// Number of pages that are skipped because of Parquet page level statistics
+ /// and HJ min/max filters.
+ RuntimeProfile::Counter* num_minmax_filtered_pages_counter_;
+
/// Number of pages need to be examined. We need to scan
/// 'num_pages_counter_ - num_stats_filtered_pages_counter_' pages.
RuntimeProfile::Counter* num_pages_counter_;
@@ -522,6 +530,58 @@ class HdfsParquetScanner : public HdfsColumnarScanner {
/// to be OK as well.
Status NextRowGroup() WARN_UNUSED_RESULT;
+ /// Evaluates the overlap predicates of the 'scan_node_' using the parquet::Statistics
+ /// of 'row_group'. 'file_metadata' is used to determine the ordering that was used to
+ /// compute the statistics. Sets 'skip_row_group' to true if the row group can be
+ /// skipped, 'false' otherwise.
+ Status EvaluateOverlapForRowGroup(
+ const parquet::FileMetaData& file_metadata, const parquet::RowGroup& row_group,
+ bool* skip_row_group);
+
+ /// Detect if a column is a collection or missing for a column chunk described by a
+ /// schema path in a slot descriptor 'slot_desc'.
+ /// On return:
+ /// Case 1: return Status::OK() and 'missing_field' is set accordingly.
+ /// 'schema_node_ptr' is optional and set to the schema node if not null.
+ //
+ /// Case 2: return a Status with an error message when the column chunk is a
+ /// collection.
+ Status ResolveSchemaForStatFiltering(SlotDescriptor* slot_desc, bool* missing_field,
+ SchemaNode** schema_node_ptr = nullptr);
+
+ /// Create a ColumnStatsReader object for a column chunk described by a schema
+ /// path in a slot descriptor 'slot_desc'. 'file_metadata', 'row_group', 'node',
+ /// and 'col_type' provide extra data needed.
+ /// On return:
+ /// A column chunk stats reader ('ColumnStatsReader') is returned.
+ ColumnStatsReader CreateStatsReader(const parquet::FileMetaData& file_metadata,
+ const parquet::RowGroup& row_group, SchemaNode* node, const ColumnType& col_type);
+
+ /// Return the overlap predicate descs from the HDFS scan plan.
+ const vector<TOverlapPredicateDesc>& GetOverlapPredicateDescs();
+
+ /// Find and return the min/max filter at filter_ctx_[filter_idx].
+ /// Return nullptr if no min/max filter is found at that location.
+ MinMaxFilter* FindMinMaxFilter(int filter_idx);
+
+ /// Return the memory addresses of the min and the max slot in min_max_tuple_ at
+ /// location overlap_slot_idx and overlap_slot_idx+1.
+ void GetMinMaxSlotsForOverlapPred(
+ int overlap_slot_idx, void** min_slot, void** max_slot);
+
+ /// Decide whether page index should be processed. Return true when
+ /// 1. Query option parquet_read_page_index is set to true, and
+ /// 2. there exist min/max conjuncts or some min/max filters from joins are available.
+ bool ShouldProcessPageIndex();
+
+ /// Find skip ranges for pages in the current row group that are outside the min/max
+ /// ranges defined by overlap predicate min/max filters. These filters are specified
+ /// by GetOverlapPredStartIndex().
+ /// Return OK if all pages have been evaluated against the filters and *'skip_ranges'
+ /// is appended with those skip ranges found (if any).
+ /// Returns a non-OK status when some error is encountered.
+ Status FindSkipRangesForPagesWithMinMaxFilters(vector<RowRange>* skip_ranges);
+
/// Resets page index filtering state, i.e. clears 'candidate_ranges_' and resets
/// scalar readers' page filtering as well.
void ResetPageFiltering();
@@ -542,6 +602,10 @@ class HdfsParquetScanner : public HdfsColumnarScanner {
/// Check that the scalar readers agree on the top-level row being scanned.
Status CheckPageFiltering();
+ /// Find out if the enabled_for_page flag at filter_stats_[filter_idx] is true. If so,
+ /// the filter at filter_ctx_[filter_idx] is worthy to evaluate the overlap predicate.
+ bool IsFilterWorthyForOverlapCheck(int filter_idx);
+
/// Reads statistics data for a page of given column. Page is specified by 'page_idx',
/// column is specified by 'column_index'. 'stats_field' specifies whether we should
/// read the min or max value. 'is_null_page' is set to true for null pages, in that
@@ -551,6 +615,15 @@ class HdfsParquetScanner : public HdfsColumnarScanner {
const parquet::ColumnIndex& column_index, int page_idx,
ColumnStatsReader::StatsField stats_field, bool* is_null_page, void* slot);
+ /// Reads both min and max statistics data for a page of given column. Page is
+ /// specified by 'page_idx', column is specified by 'column_index'. 'is_null_page'
+ /// is set to true for null pages, in that case there is no min nor max value.
+ /// Returns true if the read of the min and the max value was successful, in this
+ /// case 'min_slot' will hold the min and 'max_slot' will hold the max value.
+ static bool ReadStatFromIndex(const ColumnStatsReader& stats_reader,
+ const parquet::ColumnIndex& column_index, int page_idx,
+ bool* is_null_page, void* min_slot, void* max_slot);
+
/// Reads data using 'column_readers' to materialize top-level tuples into 'row_batch'.
/// Returns a non-OK status if a non-recoverable error was encountered and execution
/// of this query should be terminated immediately.
diff --git a/be/src/exec/parquet/parquet-column-stats.cc b/be/src/exec/parquet/parquet-column-stats.cc
index 6073860..455422e 100644
--- a/be/src/exec/parquet/parquet-column-stats.cc
+++ b/be/src/exec/parquet/parquet-column-stats.cc
@@ -84,6 +84,11 @@ bool ColumnStatsReader::ReadFromThrift(StatsField stats_field, void* slot) const
return ReadFromString(stats_field, *stat_value, paired_stats_value, slot);
}
+bool ColumnStatsReader::ReadMinMaxFromThrift(void* min_slot, void* max_slot) const {
+ return ReadFromThrift(ColumnStatsReader::StatsField::MIN, min_slot) &&
+ ReadFromThrift(ColumnStatsReader::StatsField::MAX, max_slot);
+}
+
bool ColumnStatsReader::ReadFromString(StatsField stats_field,
const string& encoded_value, const string* paired_stats_value, void* slot) const {
switch (col_type_.type) {
@@ -212,6 +217,17 @@ bool ColumnStatsReader::ReadNullCountStat(int64_t* null_count) const {
return false;
}
+bool ColumnStatsReader::AllNulls(bool* all_nulls) const {
+ DCHECK(all_nulls);
+ int64_t null_count = 0;
+ bool null_count_read = ReadNullCountStat(&null_count);
+ if (!null_count_read) {
+ return false;
+ }
+ *all_nulls = (null_count == col_chunk_.meta_data.num_values);
+ return true;
+}
+
Status ColumnStatsBase::CopyToBuffer(StringBuffer* buffer, StringValue* value) {
if (value->ptr == buffer->buffer()) return Status::OK();
buffer->Clear();
diff --git a/be/src/exec/parquet/parquet-column-stats.h b/be/src/exec/parquet/parquet-column-stats.h
index 89d1ad6..c596d2e 100644
--- a/be/src/exec/parquet/parquet-column-stats.h
+++ b/be/src/exec/parquet/parquet-column-stats.h
@@ -270,6 +270,10 @@ public:
/// was successful, false otherwise.
bool ReadFromThrift(StatsField stats_field, void* slot) const;
+ /// Call the above ReadFromThrift() for both the MIN and MAX stats. Return true if
+ /// both stats are read successfully, false otherwise.
+ bool ReadMinMaxFromThrift(void* min_slot, void* max_slot) const;
+
/// Read plain encoded value from a string 'encoded_value' into 'slot'.
/// Set paired_stats_value as nullptr if there is no corresponding paired stats,
/// or paired stats value is not set.
@@ -282,6 +286,13 @@ public:
// Returns true if the null_count stats were read successfully, false otherwise.
bool ReadNullCountStat(int64_t* null_count) const;
+ /// Gets the null_count and num_values statistics from the column chunk's metadata
+ /// and decide whether all values are nulls.
+ /// Returns true if the stats were read successfully and 'all_nulls' is set to
+ /// null_count being equal to num_values. Return false if stats can not be read. In
+ /// the latter case, 'all_nulls' is not altered.
+ bool AllNulls(bool* all_nulls) const;
+
/// Returns the required stats field for the given function. 'fn_name' can be 'le',
/// 'lt', 'ge', and 'gt' (i.e. binary operators <=, <, >=, >). If we want to check that
/// whether a column contains a value less than a constant, we need the minimum value of
diff --git a/be/src/exec/partitioned-hash-join-builder.cc b/be/src/exec/partitioned-hash-join-builder.cc
index 28d1fc5..ee21e7e 100644
--- a/be/src/exec/partitioned-hash-join-builder.cc
+++ b/be/src/exec/partitioned-hash-join-builder.cc
@@ -936,6 +936,12 @@ void PhjBuilder::PublishRuntimeFilters(int64_t num_build_rows) {
runtime_state_->filter_bank()->UpdateFilterFromLocal(
ctx.filter->id(), bloom_filter, ctx.local_min_max_filter);
+
+ if ( ctx.local_min_max_filter != nullptr ) {
+ VLOG(3) << "HJBuilder published min/max filter: "
+ << " id=" << ctx.filter->id()
+ << ", details=" << ctx.local_min_max_filter->DebugString();
+ }
}
if (filter_ctxs_.size() > 0) {
diff --git a/be/src/exec/scan-node.cc b/be/src/exec/scan-node.cc
index 7915d82..0ece19f 100644
--- a/be/src/exec/scan-node.cc
+++ b/be/src/exec/scan-node.cc
@@ -157,7 +157,7 @@ Status ScanNode::Prepare(RuntimeState* state) {
filter_ctx.filter = state->filter_bank()->RegisterConsumer(filter_desc);
// TODO: Enable stats for min-max filters when Kudu exposes info about filters
// (KUDU-2162).
- if (filter_ctx.filter->is_bloom_filter()) {
+ if (filter_ctx.filter->is_bloom_filter() || filter_ctx.filter->is_min_max_filter()) {
string filter_profile_title = Substitute("Filter $0 ($1)", filter_desc.filter_id,
PrettyPrinter::Print(filter_ctx.filter->filter_size(), TUnit::BYTES));
RuntimeProfile* profile =
diff --git a/be/src/runtime/coordinator.cc b/be/src/runtime/coordinator.cc
index 5d72026..015afbb 100644
--- a/be/src/runtime/coordinator.cc
+++ b/be/src/runtime/coordinator.cc
@@ -47,6 +47,7 @@
#include "runtime/query-driver.h"
#include "runtime/query-exec-mgr.h"
#include "runtime/query-state.h"
+#include "runtime/raw-value.h"
#include "scheduling/admission-control-client.h"
#include "scheduling/scheduler.h"
#include "service/client-request-state.h"
@@ -591,6 +592,10 @@ string Coordinator::FilterDebugString() {
table_printer.AddColumn("Enabled", false);
table_printer.AddColumn("Bloom Size", false);
table_printer.AddColumn("Est fpp", false);
+ table_printer.AddColumn("Min value", false);
+ table_printer.AddColumn("Max value", false);
+ ObjectPool temp_object_pool;
+ MemTracker temp_mem_tracker;
for (auto& v: filter_routing_table_->id_to_filter) {
vector<string> row;
const FilterState& state = v.second;
@@ -628,7 +633,7 @@ string Coordinator::FilterDebugString() {
// updates have been successfully received.
row.push_back(state.enabled() || state.received_all_updates() ? "true" : "false");
- // Add size and fpp for bloom filters, the filter type otherwise.
+ // Add size and fpp for bloom filters.
if (state.is_bloom_filter()) {
int64_t filter_size = state.desc().filter_size_bytes;
row.push_back(PrettyPrinter::Print(filter_size, TUnit::BYTES));
@@ -637,12 +642,41 @@ string Coordinator::FilterDebugString() {
stringstream ss;
ss << setprecision(3) << fpp;
row.push_back(ss.str());
+ row.push_back("");
+ row.push_back("");
} else {
+ // Add the filter type for minmax filters.
row.push_back(PrintThriftEnum(state.desc().type));
row.push_back("");
+
+ // Also add the min/max value for partitioned joins, when all updates are available
+ // and the filter is not always false. For broadcast joins, no min/max values can
+ // be shown as the filter is moved and not available in state.min_max_filter_.
+ if (state.received_all_updates()) {
+ const TRuntimeFilterDesc& desc = state.desc();
+ const TExpr& src_expr = desc.src_expr;
+ ColumnType src_type = ColumnType::FromThrift(src_expr.nodes[0].type);
+ const MinMaxFilterPB& minmax_filterPB =
+ const_cast<FilterState*>(&state)->min_max_filter();
+ if (!minmax_filterPB.always_true() && !minmax_filterPB.always_false()) {
+ MinMaxFilter* min_max_filter = MinMaxFilter::Create(
+ minmax_filterPB, src_type, &temp_object_pool, &temp_mem_tracker);
+ row.push_back(
+ RawValue::PrintValue(min_max_filter->GetMin(), src_type, src_type.scale));
+ row.push_back(
+ RawValue::PrintValue(min_max_filter->GetMax(), src_type, src_type.scale));
+ } else {
+ row.push_back("");
+ row.push_back("");
+ }
+ } else {
+ row.push_back("");
+ row.push_back("");
+ }
}
table_printer.AddRow(row);
}
+ temp_mem_tracker.Close();
// Add a line break, as in all contexts this is called we need to start a new line to
// print it correctly.
return Substitute("\n$0", table_printer.ToString());
@@ -1353,7 +1387,7 @@ vector<NetworkAddressPB> Coordinator::GetActiveBackends(
}
void Coordinator::UpdateFilter(const UpdateFilterParamsPB& params, RpcContext* context) {
- VLOG(2) << "UpdateFilter(filter_id=" << params.filter_id() << ")";
+ VLOG(2) << "Coordinator::UpdateFilter(filter_id=" << params.filter_id() << ")";
shared_lock<shared_mutex> lock(filter_routing_table_->lock);
DCHECK_NE(filter_mode_, TRuntimeFilterMode::OFF)
<< "UpdateFilter() called although runtime filters are disabled";
diff --git a/be/src/runtime/date-value.cc b/be/src/runtime/date-value.cc
index ed783b1..f144368 100644
--- a/be/src/runtime/date-value.cc
+++ b/be/src/runtime/date-value.cc
@@ -366,6 +366,24 @@ DateValue DateValue::AddYears(int64_t years) const {
return DateValue(result_year, today.month(), today.day());
}
+DateValue DateValue::SubtractDays(int64_t days) const {
+ if (UNLIKELY(!IsValid())) {
+ return DateValue();
+ }
+ int64_t delta = days_since_epoch_ - days;
+ if (delta < MIN_DAYS_SINCE_EPOCH || delta > MAX_DAYS_SINCE_EPOCH) {
+ return DateValue();
+ }
+ return DateValue(delta);
+}
+
+DateValue DateValue::FindMiddleDate(const DateValue& min, const DateValue& max) {
+ if (UNLIKELY(!min.IsValid() || !max.IsValid())) {
+ return DateValue();
+ }
+ return DateValue((min.days_since_epoch_ + max.days_since_epoch_) / 2);
+}
+
bool DateValue::ToDaysSinceEpoch(int32_t* days) const {
DCHECK(days != nullptr);
if (UNLIKELY(!IsValid())) return false;
diff --git a/be/src/runtime/date-value.h b/be/src/runtime/date-value.h
index 8351252..d260dc5 100644
--- a/be/src/runtime/date-value.h
+++ b/be/src/runtime/date-value.h
@@ -138,6 +138,13 @@ class DateValue {
/// Otherwise, return an invalid DateValue instance.
DateValue AddYears(int64_t years) const;
+ /// If this DateValue instance valid, subtract 'days' days from it and return the
+ /// result. Otherwise, return an invalid DateValue instance.
+ DateValue SubtractDays(int64_t days) const;
+
+ /// Find a middle point date between 'min' and 'max'.
+ static DateValue FindMiddleDate(const DateValue& min, const DateValue& max);
+
/// If this DateValue instance is valid, convert it to the number of days since epoch
/// and return true. Result is placed in 'days'.
/// Otherwise, return false.
diff --git a/be/src/runtime/raw-value.h b/be/src/runtime/raw-value.h
index 81250bf..d2251a5 100644
--- a/be/src/runtime/raw-value.h
+++ b/be/src/runtime/raw-value.h
@@ -56,6 +56,14 @@ class RawValue {
static void PrintValue(const void* value, const ColumnType& type, int scale,
std::string* str);
+ /// Return ascii value string.
+ static std::string PrintValue(const void* value, const ColumnType& type, int scale) {
+ std::string str;
+ PrintValue(value, type, scale, &str);
+ return str;
+ }
+
+
/// Writes the byte representation of a value to a stringstream character-by-character
static void PrintValueAsBytes(const void* value, const ColumnType& type,
std::stringstream* stream);
diff --git a/be/src/runtime/runtime-filter-ir.cc b/be/src/runtime/runtime-filter-ir.cc
index dbf58fb..4fb8eb3 100644
--- a/be/src/runtime/runtime-filter-ir.cc
+++ b/be/src/runtime/runtime-filter-ir.cc
@@ -16,14 +16,26 @@
// under the License.
#include "runtime/runtime-filter.h"
+#include "util/min-max-filter.h"
using namespace impala;
bool IR_ALWAYS_INLINE RuntimeFilter::Eval(
void* val, const ColumnType& col_type) const noexcept {
- DCHECK(is_bloom_filter());
- if (bloom_filter_.Load() == BloomFilter::ALWAYS_TRUE_FILTER) return true;
- uint32_t h = RawValue::GetHashValueFastHash32(
- val, col_type, RuntimeFilterBank::DefaultHashSeed());
- return bloom_filter_.Load()->Find(h);
+ if (LIKELY((is_bloom_filter()))) {
+ if (bloom_filter_.Load() == BloomFilter::ALWAYS_TRUE_FILTER) return true;
+ uint32_t h = RawValue::GetHashValueFastHash32(
+ val, col_type, RuntimeFilterBank::DefaultHashSeed());
+ return bloom_filter_.Load()->Find(h);
+ } else {
+ DCHECK(is_min_max_filter());
+ // Min/max overlap does not deal with nulls (val==nullptr).
+ if (LIKELY(val)) {
+ MinMaxFilter* filter = get_min_max(); // get the loaded version.
+ if (LIKELY(filter)) {
+ return filter->EvalOverlap(col_type, val, val);
+ }
+ }
+ }
+ return true;
}
diff --git a/be/src/runtime/string-value-test.cc b/be/src/runtime/string-value-test.cc
index 3cdc1f3..2eca806 100644
--- a/be/src/runtime/string-value-test.cc
+++ b/be/src/runtime/string-value-test.cc
@@ -119,5 +119,17 @@ TEST(StringValueTest, TestCompare) {
EXPECT_EQ(chars[3].ptr[3], '4');
}
+TEST(StringValueTest, TestConvertToUInt64) {
+ // Test converting StringValues to uint64_t which utilizes up to first 8 bytes.
+ EXPECT_EQ(StringValue("").ToUInt64(), 0);
+ EXPECT_EQ(StringValue("\1").ToUInt64(), 0x100000000000000);
+ EXPECT_EQ(StringValue("\1\2").ToUInt64(), 0x102000000000000);
+ EXPECT_EQ(StringValue("\1\2\3").ToUInt64(), 0x102030000000000);
+
+ // extra character does not change the result
+ EXPECT_EQ(StringValue("\1\2\3\4\5\6\7\7").ToUInt64(), 0x102030405060707);
+ EXPECT_EQ(StringValue("\1\2\3\4\5\6\7\7\7").ToUInt64(), 0x102030405060707);
+}
+
}
diff --git a/be/src/runtime/string-value.cc b/be/src/runtime/string-value.cc
index 8f747e7..ba6c266 100644
--- a/be/src/runtime/string-value.cc
+++ b/be/src/runtime/string-value.cc
@@ -32,4 +32,16 @@ ostream& operator<<(ostream& os, const StringValue& string_value) {
return os << string_value.DebugString();
}
+uint64_t StringValue::ToUInt64() const {
+ unsigned char bytes[8];
+ *((uint64_t*)bytes) = 0;
+ int chars_to_copy = (len < 8) ? len : 8;
+ for (int i = 0; i < chars_to_copy; i++) {
+ bytes[i] = static_cast<unsigned char>(ptr[i]);
+ }
+ return static_cast<uint64_t>(bytes[0]) << 56 | static_cast<uint64_t>(bytes[1]) << 48
+ | static_cast<uint64_t>(bytes[2]) << 40 | static_cast<uint64_t>(bytes[3]) << 32
+ | static_cast<uint64_t>(bytes[4]) << 24 | static_cast<uint64_t>(bytes[5]) << 16
+ | static_cast<uint64_t>(bytes[6]) << 8 | static_cast<uint64_t>(bytes[7]);
+}
}
diff --git a/be/src/runtime/string-value.h b/be/src/runtime/string-value.h
index e0e624b..37315f1 100644
--- a/be/src/runtime/string-value.h
+++ b/be/src/runtime/string-value.h
@@ -110,6 +110,10 @@ struct __attribute__((__packed__)) StringValue {
*sv = impala_udf::StringVal(reinterpret_cast<uint8_t*>(ptr), len);
}
+ // Treat up to first 8 bytes of the string as an 64-bit unsigned integer. If len is
+ // less than 8, 8-len number of bytes of value '\0' are appended.
+ uint64_t ToUInt64() const;
+
static StringValue FromStringVal(const impala_udf::StringVal& sv) {
return StringValue(reinterpret_cast<char*>(sv.ptr), sv.len);
}
diff --git a/be/src/runtime/timestamp-value.cc b/be/src/runtime/timestamp-value.cc
index 6c8adab..52565cb 100644
--- a/be/src/runtime/timestamp-value.cc
+++ b/be/src/runtime/timestamp-value.cc
@@ -214,4 +214,34 @@ string TimestampValue::ToString() const {
return ss.str();
}
+TimestampValue TimestampValue::Add(const boost::posix_time::time_duration& t) const {
+ if (!IsValidTime(t) || !HasDateAndTime()) return TimestampValue();
+
+ int64_t this_in_nano_sec = time_.total_nanoseconds();
+ int64_t t_in_nano_sec = t.total_nanoseconds();
+
+ if (this_in_nano_sec + t_in_nano_sec < NANOS_PER_DAY) {
+ return TimestampValue(date_, time_ + t);
+ } else {
+ int64_t total_in_nano_sec = this_in_nano_sec + t_in_nano_sec;
+ int64_t days = total_in_nano_sec / NANOS_PER_DAY;
+ int64_t nano_secs_remaining = total_in_nano_sec % NANOS_PER_DAY;
+ return TimestampValue(date_ + boost::gregorian::date_duration(days),
+ boost::posix_time::time_duration(0, 0, 0, nano_secs_remaining));
+ }
+}
+
+TimestampValue TimestampValue::Subtract(const boost::posix_time::time_duration& t) const {
+ if (!IsValidTime(t) || !HasDateAndTime()) return TimestampValue();
+
+ int64_t this_in_nano_sec = time_.total_nanoseconds();
+ int64_t t_in_nano_sec = t.total_nanoseconds();
+ if (this_in_nano_sec - t_in_nano_sec >= 0) {
+ return TimestampValue(date_, time_ - t);
+ } else {
+ return TimestampValue(date_ - boost::gregorian::date_duration(1),
+ boost::posix_time::time_duration(
+ 0, 0, 0, NANOS_PER_DAY + this_in_nano_sec - t_in_nano_sec));
+ }
+}
}
diff --git a/be/src/runtime/timestamp-value.h b/be/src/runtime/timestamp-value.h
index 279940f..1150e2b 100644
--- a/be/src/runtime/timestamp-value.h
+++ b/be/src/runtime/timestamp-value.h
@@ -200,7 +200,6 @@ class TimestampValue {
/// Verifies that the time is not negative and is less than a whole day.
static inline bool IsValidTime(const boost::posix_time::time_duration& time) {
- static const int64_t NANOS_PER_DAY = 1'000'000'000LL * SECONDS_PER_DAY;
return !time.is_negative()
&& time.total_nanoseconds() < NANOS_PER_DAY;
}
@@ -280,6 +279,14 @@ class TimestampValue {
const boost::gregorian::date& date() const { return date_; }
const boost::posix_time::time_duration& time() const { return time_; }
+ /// Return this added with a time duration when t is valid and HasDateAndTime() is
+ /// true on this. Return an empty TimestampValue object othewise.
+ TimestampValue Add(const boost::posix_time::time_duration& t) const;
+
+ /// Return this subtracted from a time duration when t is valid and HasDateAndTime()
+ /// is true on this. Return an empty TimestampValue object othewise.
+ TimestampValue Subtract(const boost::posix_time::time_duration& t) const;
+
TimestampValue& operator=(const boost::posix_time::ptime& ptime) {
date_ = ptime.date();
time_ = ptime.time_of_day();
@@ -341,6 +348,8 @@ class TimestampValue {
static const uint64_t SECONDS_PER_DAY = 24 * 60 * 60;
+ static const int64_t NANOS_PER_DAY = 1'000'000'000LL * SECONDS_PER_DAY;
+
/// Boost ptime leaves a gap in the structure, so we swap the order to make it
/// 12 contiguous bytes. We then must convert to and from the boost ptime data type.
/// See IMP-87 for more information on why using ptime with the 4 byte gap is
diff --git a/be/src/service/query-options.cc b/be/src/service/query-options.cc
index 172a6e8..9960fd3 100644
--- a/be/src/service/query-options.cc
+++ b/be/src/service/query-options.cc
@@ -404,6 +404,26 @@ Status impala::SetQueryOption(const string& key, const string& value,
case TImpalaQueryOptions::DISABLE_ROW_RUNTIME_FILTERING:
query_options->__set_disable_row_runtime_filtering(IsTrue(value));
break;
+ case TImpalaQueryOptions::MINMAX_FILTERING_LEVEL:
+ // Parse the enabled runtime filter types and validate it.
+ TMinmaxFilteringLevel::type enum_type;
+ RETURN_IF_ERROR(GetThriftEnum(value, "minmax filter level",
+ _TMinmaxFilteringLevel_VALUES_TO_NAMES, &enum_type));
+ query_options->__set_minmax_filtering_level(enum_type);
+ break;
+ case TImpalaQueryOptions::MINMAX_FILTER_THRESHOLD: {
+ StringParser::ParseResult status;
+ double val = StringParser::StringToFloat<double>(
+ value.c_str(), static_cast<int>(value.size()), &status);
+ if (status != StringParser::PARSE_SUCCESS || val < 0.0 || val > 1.0) {
+ return Status(
+ Substitute("Invalid minmax filter threshold '$0'. Valid sizes are in"
+ "[0.0, 1.0]",
+ value));
+ }
+ query_options->__set_minmax_filter_threshold(val);
+ break;
+ }
case TImpalaQueryOptions::MAX_NUM_RUNTIME_FILTERS: {
StringParser::ParseResult status;
int val = StringParser::StringToInt<int>(value.c_str(), value.size(), &status);
diff --git a/be/src/service/query-options.h b/be/src/service/query-options.h
index b504177..cc794e4 100644
--- a/be/src/service/query-options.h
+++ b/be/src/service/query-options.h
@@ -47,7 +47,7 @@ typedef std::unordered_map<string, beeswax::TQueryOptionLevel::type>
// time we add or remove a query option to/from the enum TImpalaQueryOptions.
#define QUERY_OPTS_TABLE\
DCHECK_EQ(_TImpalaQueryOptions_VALUES_TO_NAMES.size(),\
- TImpalaQueryOptions::ANALYTIC_RANK_PUSHDOWN_THRESHOLD + 1);\
+ TImpalaQueryOptions::MINMAX_FILTERING_LEVEL + 1);\
REMOVED_QUERY_OPT_FN(abort_on_default_limit_exceeded, ABORT_ON_DEFAULT_LIMIT_EXCEEDED)\
QUERY_OPT_FN(abort_on_error, ABORT_ON_ERROR, TQueryOptionLevel::REGULAR)\
REMOVED_QUERY_OPT_FN(allow_unsupported_formats, ALLOW_UNSUPPORTED_FORMATS)\
@@ -234,6 +234,10 @@ typedef std::unordered_map<string, beeswax::TQueryOptionLevel::type>
QUERY_OPT_FN(utf8_mode, UTF8_MODE, TQueryOptionLevel::DEVELOPMENT)\
QUERY_OPT_FN(analytic_rank_pushdown_threshold,\
ANALYTIC_RANK_PUSHDOWN_THRESHOLD, TQueryOptionLevel::ADVANCED)\
+ QUERY_OPT_FN(minmax_filter_threshold, MINMAX_FILTER_THRESHOLD,\
+ TQueryOptionLevel::ADVANCED)\
+ QUERY_OPT_FN(minmax_filtering_level, MINMAX_FILTERING_LEVEL,\
+ TQueryOptionLevel::ADVANCED)\
;
/// Enforce practical limits on some query options to avoid undesired query state.
diff --git a/be/src/util/debug-util.cc b/be/src/util/debug-util.cc
index af10ddd..1ca8fd6 100644
--- a/be/src/util/debug-util.cc
+++ b/be/src/util/debug-util.cc
@@ -102,6 +102,7 @@ PRINT_THRIFT_ENUM_IMPL(TUnit)
PRINT_THRIFT_ENUM_IMPL(TParquetTimestampType)
PRINT_THRIFT_ENUM_IMPL(TTransactionalType)
PRINT_THRIFT_ENUM_IMPL(TEnabledRuntimeFilterTypes)
+PRINT_THRIFT_ENUM_IMPL(TMinmaxFilteringLevel)
string PrintId(const TUniqueId& id, const string& separator) {
stringstream out;
diff --git a/be/src/util/debug-util.h b/be/src/util/debug-util.h
index 5852c82..e477acb 100644
--- a/be/src/util/debug-util.h
+++ b/be/src/util/debug-util.h
@@ -79,6 +79,7 @@ std::string PrintThriftEnum(const TUnit::type& value);
std::string PrintThriftEnum(const TParquetTimestampType::type& value);
std::string PrintThriftEnum(const TTransactionalType::type& value);
std::string PrintThriftEnum(const TEnabledRuntimeFilterTypes::type& value);
+std::string PrintThriftEnum(const TMinmaxFilteringLevel::type& value);
std::string PrintTuple(const Tuple* t, const TupleDescriptor& d);
std::string PrintRow(TupleRow* row, const RowDescriptor& d);
diff --git a/be/src/util/min-max-filter-ir.cc b/be/src/util/min-max-filter-ir.cc
index 524fa84..c8228b2 100644
--- a/be/src/util/min-max-filter-ir.cc
+++ b/be/src/util/min-max-filter-ir.cc
@@ -23,12 +23,12 @@ using std::string;
namespace impala {
-#define NUMERIC_MIN_MAX_FILTER_INSERT(NAME, TYPE) \
- void NAME##MinMaxFilter::Insert(const void* val) { \
- if (val == nullptr) return; \
- const TYPE* value = reinterpret_cast<const TYPE*>(val); \
- if (*value < min_) min_ = *value; \
- if (*value > max_) max_ = *value; \
+#define NUMERIC_MIN_MAX_FILTER_INSERT(NAME, TYPE) \
+ void NAME##MinMaxFilter::Insert(const void* val) { \
+ if (val == nullptr) return; \
+ const TYPE* value = reinterpret_cast<const TYPE*>(val); \
+ if (*value < min_) min_ = *value; \
+ if (*value > max_) max_ = *value; \
}
NUMERIC_MIN_MAX_FILTER_INSERT(Bool, bool);
diff --git a/be/src/util/min-max-filter-test.cc b/be/src/util/min-max-filter-test.cc
index 27c4957..1e32d02 100644
--- a/be/src/util/min-max-filter-test.cc
+++ b/be/src/util/min-max-filter-test.cc
@@ -37,8 +37,8 @@ TEST(MinMaxFilterTest, TestBoolMinMaxFilter) {
MemTracker mem_tracker;
ObjectPool obj_pool;
- MinMaxFilter* filter = MinMaxFilter::Create(
- ColumnType(PrimitiveType::TYPE_BOOLEAN), &obj_pool, &mem_tracker);
+ ColumnType bool_column_type(PrimitiveType::TYPE_BOOLEAN);
+ MinMaxFilter* filter = MinMaxFilter::Create(bool_column_type, &obj_pool, &mem_tracker);
EXPECT_TRUE(filter->AlwaysFalse());
bool b1 = true;
filter->Insert(&b1);
@@ -47,10 +47,27 @@ TEST(MinMaxFilterTest, TestBoolMinMaxFilter) {
EXPECT_FALSE(filter->AlwaysFalse());
bool b2 = false;
+
+ // The range is currently [true, true] in filter.
+ // Check overlapping with [false, false], which should be false.
+ EXPECT_EQ(filter->EvalOverlap(bool_column_type, &b2, &b2), false);
+ // Check overlapping with [true, true], which should be true.
+ EXPECT_EQ(filter->EvalOverlap(bool_column_type, &b1, &b1), true);
+ // Check overlapping with [false, true], which should be true.
+ EXPECT_EQ(filter->EvalOverlap(bool_column_type, &b2, &b1), true);
+
+ // The range is currently [false, true] in filter.
filter->Insert(&b2);
EXPECT_EQ(*reinterpret_cast<const bool*>(filter->GetMin()), b2);
EXPECT_EQ(*reinterpret_cast<const bool*>(filter->GetMax()), b1);
+ // Check overlapping with [false, false], which should be true.
+ EXPECT_EQ(filter->EvalOverlap(bool_column_type, &b2, &b2), true);
+ // Check overlapping with [true, true], which should be true.
+ EXPECT_EQ(filter->EvalOverlap(bool_column_type, &b1, &b1), true);
+ // Check overlapping with [false, true], which should be true.
+ EXPECT_EQ(filter->EvalOverlap(bool_column_type, &b2, &b1), true);
+
// Check the behavior of Or.
MinMaxFilterPB pFilter1;
pFilter1.mutable_min()->set_bool_val(false);
@@ -82,6 +99,23 @@ void CheckIntVals(MinMaxFilter* filter, int32_t min, int32_t max) {
EXPECT_EQ(*reinterpret_cast<const int32_t*>(filter->GetMax()), max);
EXPECT_FALSE(filter->AlwaysFalse());
EXPECT_FALSE(filter->AlwaysTrue());
+
+ // Check overlaps. The range is [min, max] in filter.
+ ColumnType int_type(PrimitiveType::TYPE_INT);
+
+ // Check overlapping with [min-10, min-1], which should be false.
+ int min_minus_10 = min - 10;
+ int min_minus_1 = min - 1;
+ EXPECT_EQ(filter->EvalOverlap(int_type, &min_minus_10, &min_minus_1), false);
+
+ // Check overlapping with [max+1, max+20], which should be false.
+ int max_plus_1 = max + 1;
+ int max_plus_20 = max + 20;
+ EXPECT_EQ(filter->EvalOverlap(int_type, &max_plus_1, &max_plus_20), false);
+
+ // Check overlapping with [min-1, min+(max-min)/2], which should be true.
+ int middle = min + (max-min)/2;
+ EXPECT_EQ(filter->EvalOverlap(int_type, &min_minus_1, &middle), true);
}
// Tests that a IntMinMaxFilter returns the expected min/max after having values
@@ -158,6 +192,28 @@ TEST(MinMaxFilterTest, TestNumericMinMaxFilter) {
f2->Close();
}
+// Make a string that is compared less than 'str'
+string make_less_than(const string& str) {
+ if (str.length() == 1) {
+ unsigned char c = str.c_str()[0];
+ if (c == 0) {
+ return string("");
+ }
+ c--;
+ return string(1, c);
+ }
+ string result(str);
+ result.pop_back();
+ return result;
+}
+
+// Make a string that is compared greater than 'str'
+string make_greater_than(const string& str) {
+ string result(str);
+ result.push_back('a');
+ return result;
+}
+
void CheckStringVals(MinMaxFilter* filter, const string& min, const string& max) {
StringValue actual_min = *reinterpret_cast<const StringValue*>(filter->GetMin());
StringValue actual_max = *reinterpret_cast<const StringValue*>(filter->GetMax());
@@ -167,6 +223,37 @@ void CheckStringVals(MinMaxFilter* filter, const string& min, const string& max)
EXPECT_EQ(actual_max, expected_max);
EXPECT_FALSE(filter->AlwaysTrue());
EXPECT_FALSE(filter->AlwaysFalse());
+
+ // Check overlaps. The range is [min, max] in filter.
+ ColumnType string_type(PrimitiveType::TYPE_STRING);
+
+ // Check overlapping with [less_than(min), less_than(min)], which should be false.
+ string less_than_str = make_less_than(min);
+ StringValue less_than(less_than_str);
+ EXPECT_EQ(filter->EvalOverlap(string_type, &less_than, &less_than), false);
+
+ // Check overlapping with [greater_than(max), greater_than(max)], which should be false.
+ string greater_than_str = make_greater_than(max);
+ StringValue greater_than(greater_than_str);
+ EXPECT_EQ(filter->EvalOverlap(string_type, &greater_than, &greater_than), false);
+
+ // Check overlapping with [less_than(min), max], which should be true.
+ EXPECT_EQ(filter->EvalOverlap(string_type, &less_than, &actual_max), true);
+
+ // Check overlapping with [min, greater_than(max)], which should be true.
+ EXPECT_EQ(filter->EvalOverlap(string_type, &actual_min, &greater_than), true);
+
+ // Test that uint64_t converted values are useful to compute overlap ratio.
+ // Note that Strings sharing a common prefix of 8 bytes long can be converted to the
+ // same value (e.g. Uint64("12345678") == Uint64("123456789")), the relationship of
+ // "<=" is tested.
+ uint64_t less_than_as_uint64 = less_than.ToUInt64();
+ uint64_t actual_min_as_uint64 = actual_min.ToUInt64();
+ uint64_t actual_max_as_uint64 = actual_max.ToUInt64();
+ uint64_t greater_than_as_uint64 = greater_than.ToUInt64();
+ EXPECT_EQ(less_than_as_uint64 <= actual_min_as_uint64, true);
+ EXPECT_EQ(actual_min_as_uint64 <= actual_max_as_uint64, true);
+ EXPECT_EQ(actual_max_as_uint64 <= greater_than_as_uint64, true);
}
// Tests that a StringMinMaxFilter returns the expected min/max after having values
@@ -368,12 +455,76 @@ static DateValue ParseSimpleDate(const char* s) {
return DateValue::ParseSimpleDateFormat(s, strlen(s), /* accept_time_toks */ true);
}
-#define DATE_TIME_CHECK_VALS(NAME, TYPE) \
- void Check##NAME##Vals(MinMaxFilter* filter, const TYPE& min, const TYPE& max) { \
- EXPECT_EQ(*reinterpret_cast<const TYPE*>(filter->GetMin()), min); \
- EXPECT_EQ(*reinterpret_cast<const TYPE*>(filter->GetMax()), max); \
- EXPECT_FALSE(filter->AlwaysFalse()); \
- EXPECT_FALSE(filter->AlwaysTrue()); \
+// Check overlaps on dates. The range is [min, max] in filter.
+void CheckOverlapForDate(MinMaxFilter* filter, const ColumnType& col_type,
+ const DateValue& min, const DateValue& max) {
+ // Check overlapping with [min-10, min-1], which should be false.
+ DateValue min_minus_1 = min.SubtractDays(1);
+ DateValue min_minus_10 = min.SubtractDays(10);
+ EXPECT_EQ(filter->EvalOverlap(col_type, &min_minus_10, &min_minus_1), false);
+
+ // Check overlapping with [max+1, max+20], which should be false.
+ DateValue max_plus_1 = max.AddDays(1);
+ DateValue max_plus_20 = max.AddDays(20);
+ EXPECT_EQ(filter->EvalOverlap(col_type, &max_plus_1, &max_plus_20), false);
+
+ // Check overlapping with [min-10, middle_day(min, max)], which should be true.
+ DateValue middle_date = DateValue::FindMiddleDate(min, max);
+ EXPECT_EQ(filter->EvalOverlap(col_type, &min_minus_10, &middle_date), true);
+
+ if (min < max) {
+ // Check overlapping with [min+1, max], which should be true.
+ DateValue min_plus_1 = min.AddDays(1);
+ EXPECT_EQ(filter->EvalOverlap(col_type, &min_plus_1, (void*)&max), true);
+ }
+}
+
+// Check overlaps on timestamps. The range is [min, max] in filter.
+void CheckOverlapForTimestamp(MinMaxFilter* filter, const ColumnType& col_type,
+ const TimestampValue& min, const TimestampValue& max) {
+ // Check overlapping with [min-10, min-1], which should be false.
+ TimestampValue min_minus_10 =
+ min.Subtract(boost::posix_time::time_duration(0, 0, 0, 10));
+ TimestampValue min_minus_1 = min.Subtract(boost::posix_time::time_duration(0, 0, 0, 1));
+ EXPECT_EQ(filter->EvalOverlap(col_type, &min_minus_10, &min_minus_1), false);
+
+ // Check overlapping with [max+1, max+20], which should be false.
+ TimestampValue max_plus_1 = max.Add(boost::posix_time::time_duration(0, 0, 0, 1));
+ TimestampValue max_plus_20 = max.Add(boost::posix_time::time_duration(0, 0, 0, 20));
+ EXPECT_EQ(filter->EvalOverlap(col_type, &max_plus_1, &max_plus_20), false);
+
+ // Check overlapping with [min-10, max], which should be true.
+ EXPECT_EQ(filter->EvalOverlap(col_type, &min_minus_10, (void*)&max), true);
+
+ if (min < max) {
+ // Check overlapping with [min+1 ns, max], which should be true.
+ TimestampValue min_plus_1 = min.Add(boost::posix_time::time_duration(0, 0, 0, 1));
+ EXPECT_EQ(filter->EvalOverlap(col_type, &min_plus_1, (void*)&max), true);
+ }
+
+ // The overlap ratio with [min, max] should be 1.
+ EXPECT_EQ(filter->ComputeOverlapRatio(col_type, (void*)&min, (void*)&max), 1.0);
+
+ // The overlap ratio with [min-10, max+20] should be less than or very close to 1.0,
+ // which is represented as 1.0.
+ EXPECT_TRUE(
+ (filter->ComputeOverlapRatio(col_type, (void*)&min_minus_10, (void*)&max_plus_20))
+ <= 1.0);
+
+ // No overlap. The overlap ratio should be 0.0.
+ EXPECT_EQ(
+ (filter->ComputeOverlapRatio(col_type, (void*)&min_minus_10, (void*)&min_minus_1)),
+ 0.0);
+}
+
+#define DATE_TIME_CHECK_VALS(NAME, TYPE) \
+ void Check##NAME##Vals(MinMaxFilter* filter, const ColumnType& col_type, \
+ const TYPE& min, const TYPE& max) { \
+ EXPECT_EQ(*reinterpret_cast<const TYPE*>(filter->GetMin()), min); \
+ EXPECT_EQ(*reinterpret_cast<const TYPE*>(filter->GetMax()), max); \
+ EXPECT_FALSE(filter->AlwaysFalse()); \
+ EXPECT_FALSE(filter->AlwaysTrue()); \
+ CheckOverlapFor##NAME(filter, col_type, min, max); \
}
DATE_TIME_CHECK_VALS(Timestamp, TimestampValue);
@@ -402,16 +553,16 @@ DATE_TIME_CHECK_VALS(Date, DateValue);
/* Now insert some stuff. */ \
TYPE t1 = ParseSimple##NAME("2000-01-01 00:00:00"); \
filter->Insert(&t1); \
- Check##NAME##Vals(filter, t1, t1); \
+ Check##NAME##Vals(filter, col_type, t1, t1); \
TYPE t2 = ParseSimple##NAME("1990-01-01 12:30:00"); \
filter->Insert(&t2); \
- Check##NAME##Vals(filter, t2, t1); \
+ Check##NAME##Vals(filter, col_type, t2, t1); \
TYPE t3 = ParseSimple##NAME("2001-04-30 05:00:00"); \
filter->Insert(&t3); \
- Check##NAME##Vals(filter, t2, t3); \
+ Check##NAME##Vals(filter, col_type, t2, t3); \
TYPE t4 = ParseSimple##NAME("2001-04-30 01:00:00"); \
filter->Insert(&t4); \
- Check##NAME##Vals(filter, t2, t3); \
+ Check##NAME##Vals(filter, col_type, t2, t3); \
/* Check Protobuf. */ \
filter->ToProtobuf(&pFilter); \
EXPECT_FALSE(pFilter.always_false()); \
@@ -420,7 +571,7 @@ DATE_TIME_CHECK_VALS(Date, DateValue);
EXPECT_EQ(TYPE::FromColumnValuePB(pFilter.max()), t3); \
MinMaxFilter* filter2 = \
MinMaxFilter::Create(pFilter, col_type, &obj_pool, &mem_tracker); \
- Check##NAME##Vals(filter2, t2, t3); \
+ Check##NAME##Vals(filter2, col_type, t2, t3); \
filter2->Close(); \
/* Check the behavior of Or. */ \
filter->ToProtobuf(&pFilter); \
@@ -448,26 +599,64 @@ TEST(MinMaxFilterTest, TestDateMinMaxFilter) {
DATE_TIME_CHECK_FUNCS(Date, DateValue, date, DATE);
}
+#define DECIMAL_ADD(SIZE, result_type) \
+ Decimal##SIZE##Value Decimal##SIZE##Add(const Decimal##SIZE##Value& value, \
+ int precision, int scale, double x, bool* overflow) { \
+ DCHECK(overflow); \
+ Decimal##SIZE##Value d = \
+ Decimal##SIZE##Value::FromDouble(precision, scale, x, false, overflow); \
+ if (*overflow) return Decimal##SIZE##Value(); \
+ return value.Add<result_type>(scale, d, scale, precision, scale, false, overflow); \
+ }
+
+DECIMAL_ADD(4, int32_t);
+DECIMAL_ADD(8, int64_t);
+DECIMAL_ADD(16, __int128_t);
+
#define DECIMAL_CHECK(SIZE) \
do { \
EXPECT_EQ(*reinterpret_cast<const Decimal##SIZE##Value*>(filter->GetMin()), min); \
EXPECT_EQ(*reinterpret_cast<const Decimal##SIZE##Value*>(filter->GetMax()), max); \
EXPECT_FALSE(filter->AlwaysFalse()); \
EXPECT_FALSE(filter->AlwaysTrue()); \
+ /* Check overlaps. The range is [min, max] in filter.*/ \
+ /* Check overlapping with [min-10, min-1], which should be false.*/ \
+ const int& precision = col_type.precision; \
+ const int& scale = col_type.scale; \
+ bool overflow = false; \
+ Decimal##SIZE##Value min_minus_10 = \
+ Decimal##SIZE##Add(min, precision, scale, -10, &overflow); \
+ EXPECT_EQ(overflow, false); \
+ Decimal##SIZE##Value min_minus_1 = \
+ Decimal##SIZE##Add(min, precision, scale, -1, &overflow); \
+ EXPECT_EQ(overflow, false); \
+ EXPECT_EQ(filter->EvalOverlap(col_type, &min_minus_10, &min_minus_1), false); \
+ /* Check overlapping with [max+1, max+20], which should be false. */ \
+ Decimal##SIZE##Value max_plus_1 = \
+ Decimal##SIZE##Add(max, precision, scale, 1, &overflow); \
+ EXPECT_EQ(overflow, false); \
+ Decimal##SIZE##Value max_plus_20 = \
+ Decimal##SIZE##Add(max, precision, scale, 20, &overflow); \
+ EXPECT_EQ(overflow, false); \
+ EXPECT_EQ(filter->EvalOverlap(col_type, &max_plus_1, &max_plus_20), false); \
+ /* Check overlapping with [min-1, max], which should be true. */ \
+ EXPECT_EQ(filter->EvalOverlap(col_type, &min_minus_10, (void*)&max), true); \
+ /* Check overlapping with [min, max+20], which should be true. */ \
+ EXPECT_EQ(filter->EvalOverlap(col_type, (void*)&min, &max_plus_20), true); \
} while (false)
-void CheckDecimalVals(
- MinMaxFilter* filter, const Decimal4Value& min, const Decimal4Value& max) {
+void CheckDecimalVals(MinMaxFilter* filter, const ColumnType& col_type,
+ const Decimal4Value& min, const Decimal4Value& max) {
DECIMAL_CHECK(4);
}
-void CheckDecimalVals(
- MinMaxFilter* filter, const Decimal8Value& min, const Decimal8Value& max) {
+void CheckDecimalVals(MinMaxFilter* filter, const ColumnType& col_type,
+ const Decimal8Value& min, const Decimal8Value& max) {
DECIMAL_CHECK(8);
}
-void CheckDecimalVals(
- MinMaxFilter* filter, const Decimal16Value& min, const Decimal16Value& max) {
+void CheckDecimalVals(MinMaxFilter* filter, const ColumnType& col_type,
+ const Decimal16Value& min, const Decimal16Value& max) {
DECIMAL_CHECK(16);
}
@@ -497,15 +686,15 @@ void CheckDecimalEmptyFilter(MinMaxFilter* filter, const ColumnType& column_type
d1##SIZE = \
Decimal##SIZE##Value::FromDouble(PRECISION, SCALE, VALUE1, true, &overflow); \
filter##SIZE->Insert(&d1##SIZE); \
- CheckDecimalVals(filter##SIZE, d1##SIZE, d1##SIZE); \
+ CheckDecimalVals(filter##SIZE, decimal##SIZE##_type, d1##SIZE, d1##SIZE); \
d2##SIZE = \
Decimal##SIZE##Value::FromDouble(PRECISION, SCALE, VALUE2, true, &overflow); \
filter##SIZE->Insert(&d2##SIZE); \
- CheckDecimalVals(filter##SIZE, d1##SIZE, d2##SIZE); \
+ CheckDecimalVals(filter##SIZE, decimal##SIZE##_type, d1##SIZE, d2##SIZE); \
d3##SIZE = \
Decimal##SIZE##Value::FromDouble(PRECISION, SCALE, VALUE3, true, &overflow); \
filter##SIZE->Insert(&d3##SIZE); \
- CheckDecimalVals(filter##SIZE, d3##SIZE, d2##SIZE); \
+ CheckDecimalVals(filter##SIZE, decimal##SIZE##_type, d3##SIZE, d2##SIZE); \
} while (false)
#define DECIMAL_CHECK_PROTOBUF(SIZE) \
@@ -517,7 +706,7 @@ void CheckDecimalEmptyFilter(MinMaxFilter* filter, const ColumnType& column_type
EXPECT_EQ(Decimal##SIZE##Value::FromColumnValuePB(pFilter##SIZE.max()), d2##SIZE); \
MinMaxFilter* filter##SIZE##2 = MinMaxFilter::Create( \
pFilter##SIZE, decimal##SIZE##_type, &obj_pool, &mem_tracker); \
- CheckDecimalVals(filter##SIZE##2, d3##SIZE, d2##SIZE); \
+ CheckDecimalVals(filter##SIZE##2, decimal##SIZE##_type, d3##SIZE, d2##SIZE); \
filter##SIZE##2->Close(); \
} while (false)
diff --git a/be/src/util/min-max-filter.cc b/be/src/util/min-max-filter.cc
index 187d8da..78fa51d 100644
--- a/be/src/util/min-max-filter.cc
+++ b/be/src/util/min-max-filter.cc
@@ -85,6 +85,22 @@ IRFunction::Type MinMaxFilter::GetInsertIRFunctionType(ColumnType column_type) {
}
}
+int64_t GetIntTypeValue(const ColumnType& type, const void* value) {
+ switch (type.type) {
+ case TYPE_TINYINT:
+ return *static_cast<const int8_t*>(value);
+ case TYPE_SMALLINT:
+ return *static_cast<const int16_t*>(value);
+ case TYPE_INT:
+ return *static_cast<const int32_t*>(value);
+ case TYPE_BIGINT:
+ return *static_cast<const int64_t*>(value);
+ default:
+ DCHECK(false) << "Not an int type: " << type;
+ }
+ return -1;
+}
+
#define NUMERIC_MIN_MAX_FILTER_FUNCS(NAME, TYPE, PROTOBUF_TYPE, PRIMITIVE_TYPE) \
const char* NAME##MinMaxFilter::LLVM_CLASS_NAME = \
"class.impala::" #NAME "MinMaxFilter"; \
@@ -116,7 +132,8 @@ IRFunction::Type MinMaxFilter::GetInsertIRFunctionType(ColumnType column_type) {
string NAME##MinMaxFilter::DebugString() const { \
stringstream out; \
out << #NAME << "MinMaxFilter(min=" << min_ << ", max=" << max_ \
- << ", always_false=" << (AlwaysFalse() ? "true" : "false") << ")"; \
+ << ", always_false=" << (AlwaysFalse() ? "true" : "false") << ")" \
+ << ", always_true=" << (AlwaysTrue() ? "true" : "false") << ")"; \
return out.str(); \
} \
void NAME##MinMaxFilter::Or(const MinMaxFilterPB& in, MinMaxFilterPB* out) { \
@@ -134,7 +151,7 @@ IRFunction::Type MinMaxFilter::GetInsertIRFunctionType(ColumnType column_type) {
void NAME##MinMaxFilter::Copy(const MinMaxFilterPB& in, MinMaxFilterPB* out) { \
out->mutable_min()->set_##PROTOBUF_TYPE##_val(in.min().PROTOBUF_TYPE##_val()); \
out->mutable_max()->set_##PROTOBUF_TYPE##_val(in.max().PROTOBUF_TYPE##_val()); \
- }
+ } \
NUMERIC_MIN_MAX_FILTER_FUNCS(Bool, bool, bool, BOOLEAN);
NUMERIC_MIN_MAX_FILTER_FUNCS(TinyInt, int8_t, byte, TINYINT);
@@ -176,43 +193,139 @@ int64_t GetIntTypeMin(const ColumnType& type) {
return -1;
}
-#define NUMERIC_MIN_MAX_FILTER_CAST(NAME) \
- bool NAME##MinMaxFilter::GetCastIntMinMax( \
- const ColumnType& type, int64_t* out_min, int64_t* out_max) { \
- int64_t type_min = GetIntTypeMin(type); \
- int64_t type_max = GetIntTypeMax(type); \
- if (min_ < type_min) { \
- *out_min = type_min; \
- } else if (min_ > type_max) { \
- return false; \
- } else { \
- *out_min = min_; \
- } \
- if (max_ > type_max) { \
- *out_max = type_max; \
- } else if (max_ < type_min) { \
- return false; \
- } else { \
- *out_max = max_; \
- } \
- return true; \
+#define NUMERIC_MIN_MAX_FILTER_CAST(NAME, TYPE) \
+ bool NAME##MinMaxFilter::GetCastIntMinMax( \
+ const ColumnType& col_type, int64_t* out_min, int64_t* out_max) { \
+ /* If the primitive type of the filter is the same as the column type, */ \
+ /* there is no chance of mis-alignment. */ \
+ if (type() == col_type.type) { \
+ *out_min = min_; \
+ *out_max = max_; \
+ return true; \
+ } \
+ int64_t type_min = GetIntTypeMin(col_type); \
+ int64_t type_max = GetIntTypeMax(col_type); \
+ if (min_ < type_min) { \
+ *out_min = type_min; \
+ } else if (min_ > type_max) { \
+ return false; \
+ } else { \
+ *out_min = min_; \
+ } \
+ if (max_ > type_max) { \
+ *out_max = type_max; \
+ } else if (max_ < type_min) { \
+ return false; \
+ } else { \
+ *out_max = max_; \
+ } \
+ return true; \
+ } \
+ bool NAME##MinMaxFilter::EvalOverlap( \
+ const ColumnType& col_type, void* data_min, void* data_max) { \
+ /* Apply an optimization when the column type and the filter type are the same */ \
+ if (type() == col_type.type) { \
+ return !( \
+ max_ < *static_cast<TYPE*>(data_min) || *static_cast<TYPE*>(data_max) < min_); \
+ } \
+ int64_t int_min; \
+ int64_t int_max; \
+ return EvalOverlap(col_type, data_min, data_max, &int_min, &int_max); \
+ } \
+ bool NAME##MinMaxFilter::EvalOverlap(const ColumnType& type, void* data_min, \
+ void* data_max, int64_t* filter_min64, int64_t* filter_max64) { \
+ if (!GetCastIntMinMax(type, filter_min64, filter_max64)) { \
+ /* If the filter min and max are not within the min and the*/ \
+ /* max of the type, then there is no chance of overlapping.*/ \
+ return false; \
+ } \
+ return !(*filter_max64 < GetIntTypeValue(type, data_min) \
+ || GetIntTypeValue(type, data_max) < *filter_min64); \
+ } \
+ float NAME##MinMaxFilter::ComputeOverlapRatio( \
+ const ColumnType& col_type, void* data_min, void* data_max) { \
+ /* Apply an optimization when the column type and the filter type are the same */ \
+ if (type() == col_type.type) { \
+ if (EvalOverlap(col_type, data_min, data_max)) { \
+ /* If the filter completely covers the data range, return 1.0 */ \
+ if (min_ <= *static_cast<TYPE*>(data_min) \
+ && *static_cast<TYPE*>(data_max) <= max_) { \
+ return 1.0; \
+ } \
+ TYPE overlap_min = std::max(*static_cast<TYPE*>(data_min), min_); \
+ TYPE overlap_max = std::min(*static_cast<TYPE*>(data_max), max_); \
+ return (float)((double)(overlap_max - overlap_min + 1) \
+ / (*static_cast<TYPE*>(data_max) - *static_cast<TYPE*>(data_min) + 1)); \
+ } else { \
+ return 0.0; \
+ } \
+ } \
+ int64_t filter_min64; \
+ int64_t filter_max64; \
+ if (EvalOverlap(col_type, data_min, data_max, &filter_min64, &filter_max64)) { \
+ int64_t data_min64 = GetIntTypeValue(col_type, data_min); \
+ int64_t data_max64 = GetIntTypeValue(col_type, data_max); \
+ /* If the filter completely covers the data range, return 1.0 */ \
+ if (filter_min64 <= data_min64 && data_max64 <= filter_max64) { \
+ return 1.0; \
+ } \
+ int64_t overlap_min = std::max(data_min64, filter_min64); \
+ int64_t overlap_max = std::min(data_max64, filter_max64); \
+ return (float)((double)(overlap_max - overlap_min + 1) \
+ / (data_max64 - data_min64 + 1)); \
+ } else { \
+ return 0.0; \
+ } \
}
-NUMERIC_MIN_MAX_FILTER_CAST(TinyInt);
-NUMERIC_MIN_MAX_FILTER_CAST(SmallInt);
-NUMERIC_MIN_MAX_FILTER_CAST(Int);
-NUMERIC_MIN_MAX_FILTER_CAST(BigInt);
+NUMERIC_MIN_MAX_FILTER_CAST(TinyInt, int8_t);
+NUMERIC_MIN_MAX_FILTER_CAST(SmallInt, int16_t);
+NUMERIC_MIN_MAX_FILTER_CAST(Int, int32_t);
+NUMERIC_MIN_MAX_FILTER_CAST(BigInt, int64_t);
+
-#define NUMERIC_MIN_MAX_FILTER_NO_CAST(NAME) \
+#define NUMERIC_MIN_MAX_FILTER_NO_CAST(NAME, TYPE) \
bool NAME##MinMaxFilter::GetCastIntMinMax( \
const ColumnType& type, int64_t* out_min, int64_t* out_max) { \
DCHECK(false) << "Casting min-max filters of type " << #NAME << " not supported."; \
return true; \
}
-NUMERIC_MIN_MAX_FILTER_NO_CAST(Bool);
-NUMERIC_MIN_MAX_FILTER_NO_CAST(Float);
-NUMERIC_MIN_MAX_FILTER_NO_CAST(Double);
+bool BoolMinMaxFilter::EvalOverlap(
+ const ColumnType& type, void* data_min, void* data_max) {
+ return !(max_ < *(bool*)data_min || *(bool*)data_max < min_);
+}
+
+float BoolMinMaxFilter::ComputeOverlapRatio(
+ const ColumnType& type, void* data_min, void* data_max) {
+ // For Booleans, if there is an overlap, then it is always 1.0.
+ return (EvalOverlap(type, data_min, data_max)) ? 1.0 : 0.0;
+}
+
+#define APPROXIMATE_NUMERIC_MIN_MAX_FILTER_EVAL_OVERLAP(NAME, TYPE) \
+ bool NAME##MinMaxFilter::EvalOverlap( \
+ const ColumnType& type, void* data_min, void* data_max) { \
+ return !(max_ < *(TYPE*)data_min || *(TYPE*)data_max < min_); \
+ } \
+ float NAME##MinMaxFilter::ComputeOverlapRatio( \
+ const ColumnType& type, void* data_min_ptr, void* data_max_ptr) { \
+ TYPE data_min = *(TYPE*)data_min_ptr; \
+ TYPE data_max = *(TYPE*)data_max_ptr; \
+ /* If the filter completely covers the data range, return 1.0 */ \
+ if (min_ <= data_min && data_max <= max_) { \
+ return 1.0; \
+ } \
+ TYPE overlap_min = std::max(min_, data_min); \
+ TYPE overlap_max = std::min(max_, data_max); \
+ return (overlap_max - overlap_min + 1) / (data_max - data_min + 1); \
+ }
+
+NUMERIC_MIN_MAX_FILTER_NO_CAST(Bool, bool);
+NUMERIC_MIN_MAX_FILTER_NO_CAST(Float, float);
+NUMERIC_MIN_MAX_FILTER_NO_CAST(Double, double);
+
+APPROXIMATE_NUMERIC_MIN_MAX_FILTER_EVAL_OVERLAP(Float, float);
+APPROXIMATE_NUMERIC_MIN_MAX_FILTER_EVAL_OVERLAP(Double, double);
// STRING
const char* StringMinMaxFilter::LLVM_CLASS_NAME = "class.impala::StringMinMaxFilter";
@@ -335,6 +448,28 @@ void StringMinMaxFilter::SetAlwaysTrue() {
max_.len = 0;
}
+bool StringMinMaxFilter::EvalOverlap(
+ const ColumnType& type, void* data_min, void* data_max) {
+ return !((*(StringValue*)data_max) < min_ || max_ < (*(StringValue*)data_min));
+}
+
+float StringMinMaxFilter::ComputeOverlapRatio(
+ const ColumnType& type, void* data_min_ptr, void* data_max_ptr) {
+ StringValue* data_min = (StringValue*)data_min_ptr;
+ StringValue* data_max = (StringValue*)data_max_ptr;
+ /* If the filter completely covers the data range, return 1.0*/
+ if (min_ <= *data_min && *data_max <= max_) {
+ return 1.0;
+ }
+ uint64_t data_min64 = data_min->ToUInt64();
+ uint64_t data_max64 = data_max->ToUInt64();
+ uint64_t filter_min64 = min_.ToUInt64();
+ uint64_t filter_max64 = max_.ToUInt64();
+ uint64_t overlap_min = std::max(filter_min64, data_min64);
+ uint64_t overlap_max = std::min(filter_max64, data_max64);
+ return (float)((double)(overlap_max - overlap_min + 1)) / (data_max64 - data_min64 + 1);
+}
+
// TIMESTAMP and DATE
#define DATE_TIME_MIN_MAX_FILTER_FUNCS(NAME, TYPE, PROTOBUF_TYPE, PRIMITIVE_TYPE) \
const char* NAME##MinMaxFilter::LLVM_CLASS_NAME = \
@@ -386,11 +521,55 @@ void StringMinMaxFilter::SetAlwaysTrue() {
void NAME##MinMaxFilter::Copy(const MinMaxFilterPB& in, MinMaxFilterPB* out) { \
out->mutable_min()->set_##PROTOBUF_TYPE##_val(in.min().PROTOBUF_TYPE##_val()); \
out->mutable_max()->set_##PROTOBUF_TYPE##_val(in.max().PROTOBUF_TYPE##_val()); \
+ } \
+ bool NAME##MinMaxFilter::EvalOverlap( \
+ const ColumnType& type, void* data_min, void* data_max) { \
+ return !((*(TYPE*)data_max) < min_ || max_ < (*(TYPE*)data_min)); \
}
DATE_TIME_MIN_MAX_FILTER_FUNCS(Timestamp, TimestampValue, timestamp, TIMESTAMP);
DATE_TIME_MIN_MAX_FILTER_FUNCS(Date, DateValue, date, DATE);
+float TimestampMinMaxFilter::ComputeOverlapRatio(
+ const ColumnType& type, void* data_min_ptr, void* data_max_ptr) {
+ TimestampValue* data_min = (TimestampValue*)data_min_ptr;
+ TimestampValue* data_max = (TimestampValue*)data_max_ptr;
+ /* If the filter completely covers the data range, return 1.0 */
+ if (min_ <= *data_min && *data_max <= max_) {
+ return 1.0;
+ }
+ int64_t data_min_in_ns = 0;
+ int64_t data_max_in_ns = 0;
+ int64_t filter_min_in_ns = 0;
+ int64_t filter_max_in_ns = 0;
+ if (!data_min->UtcToUnixTimeLimitedRangeNanos(&data_min_in_ns)
+ || !data_max->UtcToUnixTimeLimitedRangeNanos(&data_max_in_ns)
+ || !min_.UtcToUnixTimeLimitedRangeNanos(&filter_min_in_ns)
+ || !max_.UtcToUnixTimeLimitedRangeNanos(&filter_max_in_ns))
+ return 1.0;
+
+ int64_t overlap_min = std::max(filter_min_in_ns, data_min_in_ns);
+ int64_t overlap_max = std::min(filter_max_in_ns, data_max_in_ns);
+ return (float)(overlap_max - overlap_min + 1) / (data_max_in_ns - data_min_in_ns + 1);
+}
+
+float DateMinMaxFilter::ComputeOverlapRatio(
+ const ColumnType& type, void* data_min_ptr, void* data_max_ptr) {
+ DateValue* data_min = (DateValue*)data_min_ptr;
+ DateValue* data_max = (DateValue*)data_max_ptr;
+ /* If the filter completely covers the data range, return 1.0 */
+ if (min_ <= *data_min && *data_max <= max_) {
+ return 1.0;
+ }
+ int32_t data_days_min = data_min->Value();
+ int32_t data_days_max = data_max->Value();
+ int32_t filter_days_min = min_.Value();
+ int32_t filter_days_max = max_.Value();
+ int32_t overlap_min = std::max(filter_days_min, data_days_min);
+ int32_t overlap_max = std::min(filter_days_max, data_days_max);
+ return (float)(overlap_max - overlap_min + 1) / (data_days_max - data_days_min + 1);
+}
+
// DECIMAL
const char* DecimalMinMaxFilter::LLVM_CLASS_NAME = "class.impala::DecimalMinMaxFilter";
#define DECIMAL_SET_MINMAX(SIZE) \
@@ -540,7 +719,73 @@ void DecimalMinMaxFilter::Copy(const MinMaxFilterPB& in, MinMaxFilterPB* out) {
out->mutable_max()->set_decimal_val(in.max().decimal_val());
}
-// MinMaxFilter
+bool DecimalMinMaxFilter::EvalOverlap(
+ const ColumnType& type, void* data_min, void* data_max) {
+ bool overlap = true;
+ switch (type.GetByteSize()) {
+ case 4:
+ overlap =
+ !((*(Decimal4Value*)data_max) < min4_ || max4_ < (*(Decimal4Value*)data_min));
+ break;
+ case 8:
+ overlap =
+ !((*(Decimal8Value*)data_max) < min8_ || max8_ < (*(Decimal8Value*)data_min));
+ break;
+ case 16:
+ overlap = !(
+ (*(Decimal16Value*)data_max) < min16_ || max16_ < (*(Decimal16Value*)data_min));
+ break;
+ default:
+ DCHECK(false) << "DecimalMinMaxFilter: Unknown decimal byte size: "
+ << type.GetByteSize();
+ }
+ return overlap;
+}
+
+float DecimalMinMaxFilter::ComputeOverlapRatio(
+ const ColumnType& type, void* data_min_ptr, void* data_max_ptr) {
+ double data_min = 0;
+ double data_max = 0;
+ double filter_min = 0;
+ double filter_max = 0;
+ VLOG(1) << "byte size=" << type.GetByteSize();
+ switch (type.GetByteSize()) {
+ case 4:
+ data_min = ((Decimal4Value*)data_min_ptr)->ToDouble(type.scale);
+ data_max = ((Decimal4Value*)data_max_ptr)->ToDouble(type.scale);
+ filter_min = min4_.ToDouble(type.scale);
+ filter_max = max4_.ToDouble(type.scale);
+ break;
+ case 8:
+ data_min = ((Decimal8Value*)data_min_ptr)->ToDouble(type.scale);
+ data_max = ((Decimal8Value*)data_max_ptr)->ToDouble(type.scale);
+ filter_min = min8_.ToDouble(type.scale);
+ filter_max = max8_.ToDouble(type.scale);
+ VLOG(1) << "data_min=" << data_min
+ << ", data_max=" << data_max
+ << ", filter_min=" << filter_min
+ << ", filter_max=" << filter_max
+ << ", scale=" << type.scale;
+ break;
+ case 16:
+ data_min = ((Decimal16Value*)data_min_ptr)->ToDouble(type.scale);
+ data_max = ((Decimal16Value*)data_max_ptr)->ToDouble(type.scale);
+ filter_min = min16_.ToDouble(type.scale);
+ filter_max = max16_.ToDouble(type.scale);
+ break;
+ default:
+ DCHECK(false) << "DecimalMinMaxFilter: Unknown decimal byte size: "
+ << type.GetByteSize();
+ }
+ /* If the filter completely covers the data range, return 1.0*/
+ if (filter_min <= data_min && data_max <= filter_max) {
+ return 1.0;
+ }
+ double overlap_min = std::max(filter_min, data_min);
+ double overlap_max = std::min(filter_max, data_max);
+ return (float)((overlap_max - overlap_min + 1) / (data_max - data_min + 1));
+}
+
bool MinMaxFilter::GetCastIntMinMax(
const ColumnType& type, int64_t* out_min, int64_t* out_max) {
DCHECK(false) << "Casting min-max filters of type " << this->type()
diff --git a/be/src/util/min-max-filter.h b/be/src/util/min-max-filter.h
index 057c6a6..9489079 100644
--- a/be/src/util/min-max-filter.h
+++ b/be/src/util/min-max-filter.h
@@ -43,6 +43,7 @@ class ObjectPool;
/// filter if the join predicate is '=' and not 'is not distinct from'.
class MinMaxFilter {
public:
+ MinMaxFilter() : always_true_(false) {}
virtual ~MinMaxFilter() {}
virtual void Close() {}
@@ -59,14 +60,16 @@ class MinMaxFilter {
virtual bool GetCastIntMinMax(
const ColumnType& type, int64_t* out_min, int64_t* out_max);
+ /// Determine whether two ranges: [data_min, data_max] and [filter_min, filter_max]
+ /// overlap. Return true if the two overlaps and false otherwise. Both data_min and
+ /// data_max are of type 'type'.
+ virtual bool EvalOverlap(const ColumnType& type, void* data_min, void* data_max) = 0;
+
virtual PrimitiveType type() = 0;
/// Add a new value, updating the current min/max.
virtual void Insert(const void* val) = 0;
- /// If true, this filter allows all rows to pass.
- virtual bool AlwaysTrue() const = 0;
-
/// If true, this filter doesn't allow any rows to pass.
virtual bool AlwaysFalse() const = 0;
@@ -104,44 +107,61 @@ class MinMaxFilter {
/// Returns the IRFunction::Type for Insert() for the given type.
static IRFunction::Type GetInsertIRFunctionType(ColumnType col_type);
+ bool AlwaysTrue() const { return always_true_; }
+
+ /// Compute and return the ratio of the filter overlapping with a range defined by
+ /// [data_min, data_max] of type 'type' as follows.
+ /// 1. If there is no overlapping, ratio = 0.0;
+ /// 2. If the filter completely covers the data range, ratio = 1.0;
+ /// 3. If the filter covers the data range to a certain degree,
+ /// ratio = (overlapped area) / (data area), where
+ /// overlapped area = min(filter_max, data_max) - max(filter_min, data_min) + 1
+ /// data area = data_max - data_min + 1
+ virtual float ComputeOverlapRatio(
+ const ColumnType& type, void* data_min, void* data_max) = 0;
+
protected:
- /// Makes this filter always return true. Only valid to call for filter implementations
- /// where AlwaysTrue() can actually return true (e.g. not TIMESTAMP).
- virtual void SetAlwaysTrue() = 0;
+ /// Makes this filter always return true.
+ virtual void SetAlwaysTrue() { always_true_ = true; }
+
+ bool always_true_;
};
-#define NUMERIC_MIN_MAX_FILTER(NAME, TYPE) \
- class NAME##MinMaxFilter : public MinMaxFilter { \
- public: \
- NAME##MinMaxFilter() { \
- min_ = std::numeric_limits<TYPE>::max(); \
- max_ = std::numeric_limits<TYPE>::lowest(); \
- } \
- NAME##MinMaxFilter(const MinMaxFilterPB& protobuf); \
- virtual ~NAME##MinMaxFilter() {} \
- virtual const void* GetMin() const override { return &min_; } \
- virtual const void* GetMax() const override { return &max_; } \
- virtual bool GetCastIntMinMax( \
- const ColumnType& type, int64_t* out_min, int64_t* out_max) override; \
- virtual PrimitiveType type() override; \
- virtual void Insert(const void* val) override; \
- virtual bool AlwaysTrue() const override { return false; } \
- virtual bool AlwaysFalse() const override { \
- return min_ == std::numeric_limits<TYPE>::max() \
- && max_ == std::numeric_limits<TYPE>::lowest(); \
- } \
- virtual void SetAlwaysTrue() override { \
- DCHECK(false) << "Numeric filters cannot be always true."; \
- } \
- virtual void ToProtobuf(MinMaxFilterPB* protobuf) const override; \
- virtual std::string DebugString() const override; \
- static void Or(const MinMaxFilterPB& in, MinMaxFilterPB* out); \
- static void Copy(const MinMaxFilterPB& in, MinMaxFilterPB* out); \
- static const char* LLVM_CLASS_NAME; \
- \
- private: \
- TYPE min_; \
- TYPE max_; \
+#define NUMERIC_MIN_MAX_FILTER(NAME, TYPE) \
+ class NAME##MinMaxFilter : public MinMaxFilter { \
+ public: \
+ NAME##MinMaxFilter() { \
+ min_ = std::numeric_limits<TYPE>::max(); \
+ max_ = std::numeric_limits<TYPE>::lowest(); \
+ } \
+ NAME##MinMaxFilter(const MinMaxFilterPB& protobuf); \
+ virtual ~NAME##MinMaxFilter() {} \
+ virtual const void* GetMin() const override { return &min_; } \
+ virtual const void* GetMax() const override { return &max_; } \
+ virtual bool GetCastIntMinMax( \
+ const ColumnType& type, int64_t* out_min, int64_t* out_max) override; \
+ bool EvalOverlap(const ColumnType& type, void* data_min, void* data_max) override; \
+ float ComputeOverlapRatio( \
+ const ColumnType& type, void* data_min, void* data_max) override; \
+ virtual PrimitiveType type() override; \
+ virtual void Insert(const void* val) override; \
+ virtual bool AlwaysFalse() const override { \
+ return min_ == std::numeric_limits<TYPE>::max() \
+ && max_ == std::numeric_limits<TYPE>::lowest(); \
+ } \
+ virtual void ToProtobuf(MinMaxFilterPB* protobuf) const override; \
+ virtual std::string DebugString() const override; \
+ static void Or(const MinMaxFilterPB& in, MinMaxFilterPB* out); \
+ static void Copy(const MinMaxFilterPB& in, MinMaxFilterPB* out); \
+ static const char* LLVM_CLASS_NAME; \
+ \
+ private: \
+ bool EvalOverlap(const ColumnType& type, void* data_min, void* data_max, \
+ int64_t* filter_min64, int64_t* filter_max64); \
+ \
+ private: \
+ TYPE min_; \
+ TYPE max_; \
};
NUMERIC_MIN_MAX_FILTER(Bool, bool);
@@ -152,14 +172,16 @@ NUMERIC_MIN_MAX_FILTER(BigInt, int64_t);
NUMERIC_MIN_MAX_FILTER(Float, float);
NUMERIC_MIN_MAX_FILTER(Double, double);
+int64_t GetIntTypeValue(const ColumnType& type, const void* value);
+
class StringMinMaxFilter : public MinMaxFilter {
public:
StringMinMaxFilter(MemTracker* mem_tracker)
- : mem_pool_(mem_tracker),
+ : MinMaxFilter(),
+ mem_pool_(mem_tracker),
min_buffer_(&mem_pool_),
max_buffer_(&mem_pool_),
- always_false_(true),
- always_true_(false) {}
+ always_false_(true) {}
StringMinMaxFilter(const MinMaxFilterPB& protobuf, MemTracker* mem_tracker);
virtual ~StringMinMaxFilter() {}
virtual void Close() override { mem_pool_.FreeAll(); }
@@ -168,10 +190,13 @@ class StringMinMaxFilter : public MinMaxFilter {
virtual const void* GetMax() const override { return &max_; }
virtual PrimitiveType type() override;
+
virtual void Insert(const void* val) override;
- virtual bool AlwaysTrue() const override { return always_true_; }
virtual bool AlwaysFalse() const override { return always_false_; }
- virtual void SetAlwaysTrue() override;
+ bool EvalOverlap(
+ const ColumnType& type, void* data_min, void* data_max) override;
+ virtual float ComputeOverlapRatio(
+ const ColumnType& type, void* data_min, void* data_max) override;
/// Copies the values pointed to by 'min_'/'max_' into 'min_buffer_'/'max_buffer_',
/// truncating them if necessary.
@@ -186,6 +211,9 @@ class StringMinMaxFilter : public MinMaxFilter {
/// Struct name in LLVM IR.
static const char* LLVM_CLASS_NAME;
+ protected:
+ virtual void SetAlwaysTrue() override;
+
private:
/// Copies the contents of 'value' into 'buffer', up to 'len', and reassignes 'value' to
/// point to 'buffer'. If an oom is hit, disables the filter by setting 'always_true_'
@@ -211,35 +239,33 @@ class StringMinMaxFilter : public MinMaxFilter {
/// True if no rows have been inserted.
bool always_false_;
- bool always_true_;
};
-#define DATE_TIME_MIN_MAX_FILTER(NAME, TYPE) \
- class NAME##MinMaxFilter : public MinMaxFilter { \
- public: \
- NAME##MinMaxFilter() { always_false_ = true; } \
- NAME##MinMaxFilter(const MinMaxFilterPB& protobuf); \
- virtual ~NAME##MinMaxFilter() {} \
- virtual const void* GetMin() const override { return &min_; } \
- virtual const void* GetMax() const override { return &max_; } \
- virtual PrimitiveType type() override; \
- virtual void Insert(const void* val) override; \
- virtual bool AlwaysTrue() const override { return false; } \
- virtual bool AlwaysFalse() const override { return always_false_; } \
- virtual void ToProtobuf(MinMaxFilterPB* protobuf) const override; \
- virtual void SetAlwaysTrue() override { \
- DCHECK(false) << #NAME << " filters cannot be always true."; \
- } \
- virtual std::string DebugString() const override; \
- static void Or(const MinMaxFilterPB& in, MinMaxFilterPB* out); \
- static void Copy(const MinMaxFilterPB& in, MinMaxFilterPB* out); \
- static const char* LLVM_CLASS_NAME; \
- \
- private: \
- TYPE min_; \
- TYPE max_; \
- /* True if no rows have been inserted. */ \
- bool always_false_; \
+#define DATE_TIME_MIN_MAX_FILTER(NAME, TYPE) \
+ class NAME##MinMaxFilter : public MinMaxFilter { \
+ public: \
+ NAME##MinMaxFilter() { always_false_ = true; } \
+ NAME##MinMaxFilter(const MinMaxFilterPB& protobuf); \
+ virtual ~NAME##MinMaxFilter() {} \
+ virtual const void* GetMin() const override { return &min_; } \
+ virtual const void* GetMax() const override { return &max_; } \
+ virtual PrimitiveType type() override; \
+ virtual void Insert(const void* val) override; \
+ virtual bool AlwaysFalse() const override { return always_false_; } \
+ bool EvalOverlap(const ColumnType& type, void* data_min, void* data_max) override; \
+ virtual float ComputeOverlapRatio( \
+ const ColumnType& type, void* data_min, void* data_max) override; \
+ virtual void ToProtobuf(MinMaxFilterPB* protobuf) const override; \
+ virtual std::string DebugString() const override; \
+ static void Or(const MinMaxFilterPB& in, MinMaxFilterPB* out); \
+ static void Copy(const MinMaxFilterPB& in, MinMaxFilterPB* out); \
+ static const char* LLVM_CLASS_NAME; \
+ \
+ private: \
+ TYPE min_; \
+ TYPE max_; \
+ /* True if no rows have been inserted. */ \
+ bool always_false_; \
};
DATE_TIME_MIN_MAX_FILTER(Timestamp, TimestampValue);
@@ -294,16 +320,16 @@ class DecimalMinMaxFilter : public MinMaxFilter {
virtual void Insert(const void* val) override;
virtual PrimitiveType type() override;
- virtual bool AlwaysTrue() const override { return false; }
virtual bool AlwaysFalse() const override { return always_false_; }
- virtual void SetAlwaysTrue() override {
- DCHECK(false) << "Decimal filters cannot be always true.";
- }
virtual void ToProtobuf(MinMaxFilterPB* protobuf) const override;
virtual std::string DebugString() const override;
static void Or(const MinMaxFilterPB& in, MinMaxFilterPB* out, int precision);
static void Copy(const MinMaxFilterPB& in, MinMaxFilterPB* out);
+ bool EvalOverlap(
+ const ColumnType& type, void* data_min, void* data_max) override;
+ virtual float ComputeOverlapRatio(
+ const ColumnType& type, void* data_min, void* data_max) override;
void Insert4(const void* val);
void Insert8(const void* val);
diff --git a/common/thrift/ImpalaInternalService.thrift b/common/thrift/ImpalaInternalService.thrift
index 0a6a7e0..fb5e628 100644
--- a/common/thrift/ImpalaInternalService.thrift
+++ b/common/thrift/ImpalaInternalService.thrift
@@ -483,6 +483,13 @@ struct TQueryOptions {
// See comment in ImpalaService.thrift
122: optional i64 analytic_rank_pushdown_threshold = 1000;
+
+ // See comment in ImpalaService.thrift
+ 123: optional double minmax_filter_threshold = 0.0;
+
+ // See comment in ImpalaService.thrift
+ 124: optional PlanNodes.TMinmaxFilteringLevel minmax_filtering_level =
+ PlanNodes.TMinmaxFilteringLevel.ROW_GROUP;
}
// Impala currently has two types of sessions: Beeswax and HiveServer2
diff --git a/common/thrift/ImpalaService.thrift b/common/thrift/ImpalaService.thrift
index 61f2fb9..66a8335 100644
--- a/common/thrift/ImpalaService.thrift
+++ b/common/thrift/ImpalaService.thrift
@@ -630,6 +630,19 @@ enum TImpalaQueryOptions {
// used in the top-n operation. The larger the heaps, the less beneficial the
// optimization is compared to a full sort and the more potential for perf regressions.
ANALYTIC_RANK_PUSHDOWN_THRESHOLD = 121
+
+ // Specifiy a threshold between 0.0 and 1.0 to control the operation of overlap
+ // predicates (via min/max filters) for equi-hash joins at the Parquet table scan
+ // operator. Set to 0.0 to disable the feature. Set to 1.0 to evaluate all
+ // overlap predicates. Set to a value between 0.0 and 1.0 to evaluate only those
+ // with an overlap ratio less than the threshold.
+ MINMAX_FILTER_THRESHOLD = 122
+
+ // Minmax filtering level to be applied to Parquet scanners.
+ // ROW_GROUP - apply to row groups only,
+ // PAGE - apply to row groups and pages only.
+ // ROW - apply to row groups, pages and rows.
+ MINMAX_FILTERING_LEVEL = 123
}
// The summary of a DML statement.
diff --git a/common/thrift/PlanNodes.thrift b/common/thrift/PlanNodes.thrift
index d93224d..8daad7a 100644
--- a/common/thrift/PlanNodes.thrift
+++ b/common/thrift/PlanNodes.thrift
@@ -126,6 +126,13 @@ enum TEnabledRuntimeFilterTypes {
ALL = 3
}
+// The level of filtering of enabled min/max filters to be applied to Parquet scan nodes.
+enum TMinmaxFilteringLevel {
+ ROW_GROUP = 1
+ PAGE = 2
+ ROW = 3
+}
+
// Specification of a runtime filter.
struct TRuntimeFilterDesc {
// Filter unique id (within a query)
@@ -254,6 +261,15 @@ struct TScanRange {
3: optional binary kudu_scan_token
}
+// Specification of an overlap predicate desc.
+// filter_id: id of the filter
+// slot_index: the index of a slot descriptor in minMaxTuple where the min value is
+// stored. The slot next to it (at index +1) stores the max value.
+struct TOverlapPredicateDesc {
+ 1: required i32 filter_id
+ 2: required i32 slot_index
+}
+
struct THdfsScanNode {
1: required Types.TTupleId tuple_id
@@ -299,6 +315,9 @@ struct THdfsScanNode {
// codegen code remotely for other impalad's as it contains all file formats that may be
// needed for this scan node.
12: required set<CatalogObjects.THdfsFileFormat> file_formats;
+
+ // The overlap predicates
+ 13: optional list<TOverlapPredicateDesc> overlap_predicate_descs
}
struct TDataSourceScanNode {
diff --git a/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java b/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java
index 95adae0..13a8f1f 100644
--- a/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java
+++ b/fe/src/main/java/org/apache/impala/analysis/BinaryPredicate.java
@@ -74,6 +74,7 @@ public class BinaryPredicate extends Predicate {
public String getName() { return name_; }
public TComparisonOp getThriftOp() { return thriftOp_; }
public boolean isEquivalence() { return this == EQ || this == NOT_DISTINCT; }
+ public boolean isSqlEquivalence() { return this == EQ; }
public Operator converse() {
switch (this) {
diff --git a/fe/src/main/java/org/apache/impala/analysis/Predicate.java b/fe/src/main/java/org/apache/impala/analysis/Predicate.java
index c704e30..e9526c7 100644
--- a/fe/src/main/java/org/apache/impala/analysis/Predicate.java
+++ b/fe/src/main/java/org/apache/impala/analysis/Predicate.java
@@ -87,6 +87,11 @@ public abstract class Predicate extends Expr {
&& ((BinaryPredicate) expr).getOp().isEquivalence();
}
+ public static boolean isSqlEquivalencePredicate(Expr expr) {
+ return (expr instanceof BinaryPredicate)
+ && ((BinaryPredicate) expr).getOp().isSqlEquivalence();
+ }
+
/**
* If predicate is of the form "<slotref> = <slotref>", returns both SlotRefs,
* otherwise returns null.
diff --git a/fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java b/fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java
index 068f64d..3f5a315 100644
--- a/fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java
+++ b/fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java
@@ -259,6 +259,10 @@ public class TupleDescriptor {
return ttupleDesc;
}
+ public void resetHasMemoryLayout() {
+ hasMemLayout_ = false;
+ }
+
/**
* In some cases changes are made to a tuple after the memory layout has been computed.
* This function allows us to recompute the memory layout, if necessary. No-op if this
diff --git a/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java b/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
index a697fc5..0d1cdff 100644
--- a/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
+++ b/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
@@ -20,6 +20,7 @@ package org.apache.impala.planner;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
+import java.util.Comparator;
import java.util.HashMap;
import java.util.HashSet;
import java.util.LinkedHashMap;
@@ -32,13 +33,16 @@ import java.util.stream.Collectors;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.impala.analysis.Analyzer;
+import org.apache.impala.analysis.AlterTableSortByStmt;
import org.apache.impala.analysis.BinaryPredicate;
+import org.apache.impala.analysis.CompoundPredicate;
import org.apache.impala.analysis.DescriptorTable;
import org.apache.impala.analysis.Expr;
import org.apache.impala.analysis.InPredicate;
import org.apache.impala.analysis.IsNotEmptyPredicate;
import org.apache.impala.analysis.LiteralExpr;
import org.apache.impala.analysis.MultiAggregateInfo;
+import org.apache.impala.analysis.Path;
import org.apache.impala.analysis.SlotDescriptor;
import org.apache.impala.analysis.SlotId;
import org.apache.impala.analysis.SlotRef;
@@ -55,8 +59,10 @@ import org.apache.impala.catalog.HdfsCompression;
import org.apache.impala.catalog.HdfsFileFormat;
import org.apache.impala.catalog.HdfsPartition.FileBlock;
import org.apache.impala.catalog.HdfsPartition.FileDescriptor;
+import org.apache.impala.catalog.PrimitiveType;
import org.apache.impala.catalog.ScalarType;
import org.apache.impala.catalog.Type;
+import org.apache.impala.common.AnalysisException;
import org.apache.impala.common.FileSystemUtil;
import org.apache.impala.common.ImpalaException;
import org.apache.impala.common.ImpalaRuntimeException;
@@ -66,6 +72,7 @@ import org.apache.impala.common.Pair;
import org.apache.impala.common.PrintUtils;
import org.apache.impala.common.RuntimeEnv;
import org.apache.impala.fb.FbFileBlock;
+import org.apache.impala.planner.RuntimeFilterGenerator.RuntimeFilter;
import org.apache.impala.service.BackendConfig;
import org.apache.impala.thrift.TExplainLevel;
import org.apache.impala.thrift.TExpr;
@@ -73,10 +80,12 @@ import org.apache.impala.thrift.TFileSplitGeneratorSpec;
import org.apache.impala.thrift.THdfsFileSplit;
import org.apache.impala.thrift.THdfsScanNode;
import org.apache.impala.thrift.TNetworkAddress;
+import org.apache.impala.thrift.TOverlapPredicateDesc;
import org.apache.impala.thrift.TPlanNode;
import org.apache.impala.thrift.TPlanNodeType;
import org.apache.impala.thrift.TQueryOptions;
import org.apache.impala.thrift.TReplicaPreference;
+import org.apache.impala.thrift.TRuntimeFilterType;
import org.apache.impala.thrift.TScanRange;
import org.apache.impala.thrift.TScanRangeLocation;
import org.apache.impala.thrift.TScanRangeLocationList;
@@ -92,6 +101,7 @@ import com.google.common.base.Joiner;
import com.google.common.base.MoreObjects;
import com.google.common.base.MoreObjects.ToStringHelper;
import com.google.common.base.Preconditions;
+import com.google.common.base.Splitter;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Lists;
import com.google.common.collect.Sets;
@@ -219,6 +229,9 @@ public class HdfsScanNode extends ScanNode {
// File formats scanned. Set in computeScanRangeLocations().
private Set<HdfsFileFormat> fileFormats_;
+ // Whether all formats scanned are Parquet. Set in computeScanRangeLocations().
+ private boolean allParquet_ = false;
+
// Number of bytes in the largest scan range (i.e. hdfs split). Set in
// computeScanRangeLocations().
private long largestScanRangeBytes_ = 0;
@@ -300,6 +313,12 @@ public class HdfsScanNode extends ScanNode {
// parquet::Statistics.
private TupleDescriptor minMaxTuple_;
+ // The list of overlap predicate descs. See TOverlapPredicateDesc in PlanNodes.thrift.
+ private ArrayList<TOverlapPredicateDesc> overlapPredicateDescs_ = new ArrayList<>();
+
+ // Index of the first slot in minMaxTuple_ for overlap predicates.
+ private int overlap_first_slot_idx_ = -1;
+
// Slot that is used to record the Parquet metadata for the count(*) aggregation if
// this scan node has the count(*) optimization enabled.
private SlotDescriptor countStarSlot_ = null;
@@ -646,6 +665,133 @@ public class HdfsScanNode extends ScanNode {
minMaxTuple_.computeMemLayout();
}
+ // Init the necessary data structures prior to the detection of overlap predicates.
+ public void initOverlapPredicate(Analyzer analyzer) {
+ if (!allParquet_) return;
+ Preconditions.checkNotNull(minMaxTuple_);
+ // Allow the tuple to accept new slots.
+ minMaxTuple_.resetHasMemoryLayout();
+
+ overlap_first_slot_idx_ = minMaxTuple_.getSlots().size();
+ }
+
+ // Data type check on the slot type and the join column type.
+ private boolean checkTypeForOverlapPredicate(Type slotType, Type joinType) {
+ // Both slotType and joinType must be Boolean at the same time.
+ if (slotType.isBoolean() && joinType.isBoolean()) {
+ return true;
+ }
+
+ // Both slotType and joinType must be one of the integer types (tinyint, smallint,
+ // int, or bigint) at the same time.
+ if (slotType.isIntegerType() && joinType.isIntegerType()) {
+ return true;
+ }
+
+ // Both slotType and joinType must be strings at the same time, as CHAR or VARCHAR
+ // are not supported by min/max filters.
+ if (slotType.isScalarType(PrimitiveType.STRING)
+ && joinType.isScalarType(PrimitiveType.STRING)) {
+ return true;
+ }
+
+ // Both slotType and joinType must be timestamp at the same time.
+ if (slotType.isTimestamp() && joinType.isTimestamp()) {
+ return true;
+ }
+
+ // Both slotType and joinType must be date at the same time.
+ if (slotType.isDate() && joinType.isDate()) {
+ return true;
+ }
+
+ // Both slotType and joinType must be approximate numeric at the same time.
+ if (slotType.isFloatingPointType() && joinType.isFloatingPointType()) {
+ return true;
+ }
+
+ // If slotType and joinType are both decimals, make sure both are of the same
+ // precision and scale. This is because the backend representation of a decimal
+ // value (DecimalValue) does not carry precision and scale with it. Without a
+ // cast, it is difficult to compare two decimals with different precisions and
+ // scales.
+ if (slotType.isDecimal() && joinType.isDecimal()) {
+ ScalarType slotScalarType = (ScalarType)slotType;
+ ScalarType hashScalarType = (ScalarType)joinType;
+ return (slotScalarType.decimalPrecision() == hashScalarType.decimalPrecision()
+ && slotScalarType.decimalScale() == hashScalarType.decimalScale());
+ }
+
+ return false;
+ }
+
+ // Try to compute the overlap predicate for the filter. Return true if an overlap
+ // predicate can be formed utilizing the min/max filter 'filter' against the
+ // target expr 'targetExpr'. Return false otherwise.
+ public Boolean tryToComputeOverlapPredicate(
+ Analyzer analyzer, RuntimeFilter filter, Expr targetExpr) {
+ // This optimization is only valid for min/max filters and Parquet tables.
+ if (filter.getType() != TRuntimeFilterType.MIN_MAX) return false;
+ if (!allParquet_) return false;
+
+ // The unwrapped targetExpr should refer to a column in the scan node.
+ SlotRef slotRefInScan = targetExpr.unwrapSlotRef(true);
+ if (slotRefInScan == null) return false;
+
+ // Check if targetExpr refers to some column in one of the min/max conjuncts
+ // already formed. If so, do not add an overlap predicate as it may not be
+ // as effective as the conjunct.
+ List<SlotDescriptor> slotDescs = minMaxTuple_.getSlots();
+ for (int i=0; i<overlap_first_slot_idx_; i++) {
+ if (slotDescs.get(i).getPath() == slotRefInScan.getDesc().getPath())
+ return false;
+ }
+
+ Expr srcExpr = filter.getSrcExpr();
+
+ // When the target is not an implicit cast, make a type check between the type
+ // of the min/max stats that overlap predicate will see from the row groups or
+ // pages, and the type of the min/max from the other side of the join.
+ if (!targetExpr.isImplicitCast()) {
+ if (!checkTypeForOverlapPredicate(slotRefInScan.getType(), srcExpr.getType())) {
+ return false;
+ }
+ } else {
+ // The target is an implicit cast, make the following two type checks to
+ // assure the overlap filter can work with the row group or page stats
+ // (without any cast) and the min/max from the other side of the join.
+ // 1. The type of the target column and the casted-to type;
+ // 2. The the casted-to type and the type of the column from the
+ // other side of the join.
+ //
+ // To do: cast the row group or page stats before overlap predicate
+ // evaluation.
+ if (!checkTypeForOverlapPredicate(slotRefInScan.getType(), targetExpr.getType())
+ || !checkTypeForOverlapPredicate(targetExpr.getType(), srcExpr.getType())) {
+ return false;
+ }
+ }
+
+ int firstSlotIdx = minMaxTuple_.getSlots().size();
+ // Make two new slot descriptors to hold data min/max values (such as from
+ // row groups or pages in Parquet)) and append them to the tuple descriptor.
+ SlotDescriptor slotDescDataMin =
+ analyzer.getDescTbl().copySlotDescriptor(minMaxTuple_, slotRefInScan.getDesc());
+ SlotDescriptor slotDescDataMax =
+ analyzer.getDescTbl().copySlotDescriptor(minMaxTuple_, slotRefInScan.getDesc());
+
+ overlapPredicateDescs_.add(
+ new TOverlapPredicateDesc(filter.getFilterId().asInt(), firstSlotIdx));
+ return true;
+ }
+
+ // Finalize the necessary data structures.
+ public void finalizeOverlapPredicate() {
+ if (!allParquet_) return;
+ // Recompute the memory layout for the min/max tuple.
+ minMaxTuple_.computeMemLayout();
+ }
+
/**
* Recursively collects and assigns conjuncts bound by tuples materialized in a
* collection-typed slot. As conjuncts are seen, collect non-empty nested collections.
@@ -842,6 +988,7 @@ public class HdfsScanNode extends ScanNode {
largestScanRangeBytes_ = 0;
maxScanRangeNumRows_ = -1;
fileFormats_ = new HashSet<>();
+ boolean allParquet = true;
long simpleLimitNumRows = 0; // only used for the simple limit case
boolean isSimpleLimit = sampleParams_ == null &&
(analyzer.getQueryCtx().client_request.getQuery_options()
@@ -891,6 +1038,9 @@ public class HdfsScanNode extends ScanNode {
analyzer.getDescTbl().addReferencedPartition(tbl_, partition.getId());
fileFormats_.add(partition.getFileFormat());
+ if (!isParquetBased(partition.getFileFormat())) {
+ allParquet = false;
+ }
Preconditions.checkState(partition.getId() >= 0);
if (!fsHasBlocks) {
@@ -953,6 +1103,7 @@ public class HdfsScanNode extends ScanNode {
break;
}
}
+ allParquet_ = allParquet;
if (totalFilesPerFs_.isEmpty() || sumValues(totalFilesPerFs_) == 0) {
maxScanRangeNumRows_ = 0;
} else {
@@ -1485,8 +1636,12 @@ public class HdfsScanNode extends ScanNode {
for (Expr e: minMaxConjuncts_) {
msg.hdfs_scan_node.addToMin_max_conjuncts(e.treeToThrift());
}
+ }
+
+ if ( minMaxTuple_ != null ) {
msg.hdfs_scan_node.setMin_max_tuple_id(minMaxTuple_.getId().asInt());
}
+
Map<Integer, List<Integer>> dictMap = new LinkedHashMap<>();
for (Map.Entry<SlotDescriptor, List<Integer>> entry :
dictionaryFilterConjuncts_.entrySet()) {
@@ -1498,6 +1653,12 @@ public class HdfsScanNode extends ScanNode {
for (HdfsFileFormat format : fileFormats_) {
msg.hdfs_scan_node.addToFile_formats(format.toThrift());
}
+
+ if (!overlapPredicateDescs_.isEmpty()) {
+ for (TOverlapPredicateDesc desc: overlapPredicateDescs_) {
+ msg.hdfs_scan_node.addToOverlap_predicate_descs(desc);
+ }
+ }
}
@Override
@@ -2001,4 +2162,22 @@ public class HdfsScanNode extends ScanNode {
return scanRangeSpecs_.getConcrete_rangesSize()
+ scanRangeSpecs_.getSplit_specsSize();
}
+
+ /**
+ * Sort filters in runtimeFilters_: min/max first followed by bloom.
+ */
+ public void arrangeRuntimefiltersForParquet() {
+ if (allParquet_) {
+ Collections.sort(runtimeFilters_, new Comparator<RuntimeFilter>() {
+ @Override
+ public int compare(RuntimeFilter a, RuntimeFilter b) {
+ if (a.getType() == b.getType()) return 0;
+ if (a.getType() == TRuntimeFilterType.MIN_MAX
+ && b.getType() == TRuntimeFilterType.BLOOM)
+ return -1;
+ return 1;
+ }
+ });
+ }
+ }
}
diff --git a/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java b/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java
index 1422129..87e9c65 100644
--- a/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java
+++ b/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java
@@ -334,12 +334,17 @@ public final class RuntimeFilterGenerator {
Preconditions.checkNotNull(joinPredicate);
Preconditions.checkNotNull(filterSrcNode);
// Only consider binary equality predicates
- if (!Predicate.isEquivalencePredicate(joinPredicate)) return null;
-
- BinaryPredicate normalizedJoinConjunct =
- SingleNodePlanner.getNormalizedEqPred(joinPredicate,
- filterSrcNode.getChild(0).getTupleIds(),
- filterSrcNode.getChild(1).getTupleIds(), analyzer);
+ if (type == TRuntimeFilterType.BLOOM
+ && !Predicate.isEquivalencePredicate(joinPredicate)) {
+ return null;
+ }
+ if (type == TRuntimeFilterType.MIN_MAX
+ && !Predicate.isSqlEquivalencePredicate(joinPredicate)) {
+ return null;
+ }
+ BinaryPredicate normalizedJoinConjunct = SingleNodePlanner.getNormalizedEqPred(
+ joinPredicate, filterSrcNode.getChild(0).getTupleIds(),
+ filterSrcNode.getChild(1).getTupleIds(), analyzer);
if (normalizedJoinConjunct == null) return null;
// Ensure that the target expr does not contain TupleIsNull predicates as these
@@ -636,6 +641,19 @@ public final class RuntimeFilterGenerator {
if (LOG.isTraceEnabled()) LOG.trace("Runtime filter: " + filter.debugString());
filter.assignToPlanNodes();
}
+
+ arrangeRuntimefiltersForParquet(plan);
+ }
+
+ public static void arrangeRuntimefiltersForParquet(
+ PlanNode root) {
+ if (root instanceof HdfsScanNode) {
+ ((HdfsScanNode)root).arrangeRuntimefiltersForParquet();
+ } else {
+ for (PlanNode childNode: root.getChildren()) {
+ arrangeRuntimefiltersForParquet(childNode);
+ }
+ }
}
/**
@@ -677,6 +695,7 @@ public final class RuntimeFilterGenerator {
joinConjuncts.addAll(joinNode.getEqJoinConjuncts());
}
joinConjuncts.addAll(joinNode.getConjuncts());
+
List<RuntimeFilter> filters = new ArrayList<>();
for (TRuntimeFilterType filterType : TRuntimeFilterType.values()) {
for (Expr conjunct : joinConjuncts) {
@@ -774,11 +793,15 @@ public final class RuntimeFilterGenerator {
* to 'scanNode' if the filter is produced within the same fragment that contains the
* scan node.
* 3. Only Hdfs and Kudu scan nodes are supported:
- * a. If the target is an HdfsScanNode, the filter must be type BLOOM.
+ * a. If the target is an HdfsScanNode, the filter must be type BLOOM for non
+ * Parquet tables, or type BLOOM and/or MIN_MAX for Parquet tables.
* b. If the target is a KuduScanNode, the filter could be type MIN_MAX, and/or
* BLOOM, the target must be a slot ref on a column, and the comp op cannot
* be 'not distinct'.
- * A scan node may be used as a destination node for multiple runtime filters.
+ * A scan node may be used as a destination node for multiple runtime filters. This
+ * method is called once per scan node to process all filters accumulated for it
+ * for the entire query, per top-down traversal nature of the calling method
+ * generateFilters().
*/
private void assignRuntimeFilters(PlannerContext ctx, ScanNode scanNode) {
if (!(scanNode instanceof HdfsScanNode || scanNode instanceof KuduScanNode)) return;
@@ -787,9 +810,17 @@ public final class RuntimeFilterGenerator {
Analyzer analyzer = ctx.getRootAnalyzer();
boolean disableRowRuntimeFiltering =
ctx.getQueryOptions().isDisable_row_runtime_filtering();
+ boolean disable_overlap_filter =
+ ctx.getQueryOptions().getMinmax_filter_threshold() == 0.0;
TRuntimeFilterMode runtimeFilterMode = ctx.getQueryOptions().getRuntime_filter_mode();
TEnabledRuntimeFilterTypes enabledRuntimeFilterTypes =
ctx.getQueryOptions().getEnabled_runtime_filter_types();
+
+ // Init the overlap predicate for the hdfs scan node.
+ if (scanNode instanceof HdfsScanNode && !disable_overlap_filter) {
+ ((HdfsScanNode) scanNode).initOverlapPredicate(analyzer);
+ }
+
for (RuntimeFilter filter: runtimeFiltersByTid_.get(tid)) {
if (filter.isFinalized()) continue;
Expr targetExpr = computeTargetExpr(filter, tid, analyzer);
@@ -802,10 +833,32 @@ public final class RuntimeFilterGenerator {
// Check that the scan node supports applying filters of this type and targetExpr.
if (scanNode instanceof HdfsScanNode) {
- if (filter.getType() != TRuntimeFilterType.BLOOM
- || filter.isTimestampTruncation()) {
+ if (filter.isTimestampTruncation()) {
continue;
}
+ if (filter.getType() == TRuntimeFilterType.MIN_MAX) {
+ boolean allow_min_max =
+ enabledRuntimeFilterTypes == TEnabledRuntimeFilterTypes.MIN_MAX
+ || enabledRuntimeFilterTypes == TEnabledRuntimeFilterTypes.ALL;
+ if (!allow_min_max) {
+ continue;
+ }
+ // TODO: Apply min/max filters on partition columns.
+ if (isBoundByPartitionColumns) {
+ continue;
+ }
+ if (!disable_overlap_filter) {
+ // If the filter is not defined on partition columns, try to compute
+ // an overlap predicate for it. This predicate will be used to filter
+ // out row groups or pages in Parquet data files.
+ if (!((HdfsScanNode) scanNode)
+ .tryToComputeOverlapPredicate(analyzer, filter, targetExpr)) {
+ continue;
+ }
+ } else {
+ continue;
+ }
+ }
} else {
Preconditions.checkState(scanNode instanceof KuduScanNode);
if (filter.getType() == TRuntimeFilterType.BLOOM) {
@@ -848,11 +901,15 @@ public final class RuntimeFilterGenerator {
}
}
}
-
RuntimeFilter.RuntimeFilterTarget target = new RuntimeFilter.RuntimeFilterTarget(
scanNode, targetExpr, isBoundByPartitionColumns, isLocalTarget);
filter.addTarget(target);
}
+
+ // finalize the overlap predicate for the hdfs scan node.
+ if (scanNode instanceof HdfsScanNode && !disable_overlap_filter) {
+ ((HdfsScanNode) scanNode).finalizeOverlapPredicate();
+ }
}
/**
diff --git a/fe/src/test/java/org/apache/impala/planner/PlannerTest.java b/fe/src/test/java/org/apache/impala/planner/PlannerTest.java
index 2849c43..b5ae17a 100644
--- a/fe/src/test/java/org/apache/impala/planner/PlannerTest.java
+++ b/fe/src/test/java/org/apache/impala/planner/PlannerTest.java
@@ -398,6 +398,7 @@ public class PlannerTest extends PlannerTestBase {
// The FK/PK detection result is included in EXTENDED or higher.
TQueryOptions options = defaultQueryOptions();
options.setDisable_hdfs_num_rows_estimate(true);
+ options.setMinmax_filter_threshold(0.0);
runPlannerTestFile("fk-pk-join-detection",
options, ImmutableSet.of(PlannerTestOption.EXTENDED_EXPLAIN,
PlannerTestOption.VALIDATE_CARDINALITY));
@@ -467,7 +468,9 @@ public class PlannerTest extends PlannerTestBase {
@Test
public void testSetOperationRewrite() {
- runPlannerTestFile("setoperation-rewrite");
+ TQueryOptions options = defaultQueryOptions();
+ options.setMinmax_filter_threshold(0.0);
+ runPlannerTestFile("setoperation-rewrite", options);
}
@Test
@@ -547,6 +550,7 @@ public class PlannerTest extends PlannerTestBase {
@Test
public void testTpchNested() {
+ TQueryOptions options = new TQueryOptions();
runPlannerTestFile("tpch-nested", "tpch_nested_parquet",
ImmutableSet.of(PlannerTestOption.INCLUDE_RESOURCE_HEADER,
PlannerTestOption.VALIDATE_RESOURCES,
@@ -598,6 +602,17 @@ public class PlannerTest extends PlannerTestBase {
}
@Test
+ public void testDisableRuntimeOverlapFilter() {
+ TQueryOptions options = new TQueryOptions();
+ options.setMinmax_filter_threshold(0.0);
+ runPlannerTestFile("disable-runtime-overlap-filter", options);
+
+ options.setMinmax_filter_threshold(1.0);
+ options.setEnabled_runtime_filter_types(TEnabledRuntimeFilterTypes.BLOOM);
+ runPlannerTestFile("disable-runtime-overlap-filter", options);
+ }
+
+ @Test
public void testRuntimeFilterQueryOptions() {
runPlannerTestFile("runtime-filter-query-options");
}
@@ -746,6 +761,8 @@ public class PlannerTest extends PlannerTestBase {
// Tests the resource requirement computation from the planner.
TQueryOptions options = defaultQueryOptions();
options.setNum_scanner_threads(1); // Required so that output doesn't vary by machine
+ options.setMinmax_filter_threshold(0.0);
+ // Required so that output doesn't vary by whether parquet tables are used or not.
options.setDisable_hdfs_num_rows_estimate(true);
runPlannerTestFile("resource-requirements", options,
ImmutableSet.of(PlannerTestOption.EXTENDED_EXPLAIN,
@@ -760,6 +777,7 @@ public class PlannerTest extends PlannerTestBase {
TQueryOptions options = defaultQueryOptions();
options.setExplain_level(TExplainLevel.EXTENDED);
options.setNum_scanner_threads(1); // Required so that output doesn't vary by machine
+ options.setMinmax_filter_threshold(0.0);
options.setDisable_hdfs_num_rows_estimate(true);
runPlannerTestFile("spillable-buffer-sizing", options,
ImmutableSet.of(PlannerTestOption.EXTENDED_EXPLAIN,
@@ -774,6 +792,8 @@ public class PlannerTest extends PlannerTestBase {
TQueryOptions options = defaultQueryOptions();
options.setExplain_level(TExplainLevel.EXTENDED);
options.setNum_scanner_threads(1); // Required so that output doesn't vary by machine
+ // Required so that output doesn't vary by the format of the table used.
+ options.setMinmax_filter_threshold(0.0);
options.setMax_row_size(8L * 1024L * 1024L);
runPlannerTestFile("max-row-size", options,
ImmutableSet.of(PlannerTestOption.EXTENDED_EXPLAIN,
@@ -1079,7 +1099,9 @@ public class PlannerTest extends PlannerTestBase {
*/
@Test
public void testConvertToCNF() {
- runPlannerTestFile("convert-to-cnf", "tpch_parquet");
+ TQueryOptions options = defaultQueryOptions();
+ options.setMinmax_filter_threshold(0.0);
+ runPlannerTestFile("convert-to-cnf", "tpch_parquet", options);
}
/**
diff --git a/fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java b/fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java
index bb53d06..e9dcdae 100644
--- a/fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java
+++ b/fe/src/test/java/org/apache/impala/planner/PlannerTestBase.java
@@ -866,7 +866,7 @@ public class PlannerTestBase extends FrontendTestBase {
Collections.<PlannerTestOption>emptySet());
}
- private void runPlannerTestFile(String testFile, String dbName, TQueryOptions options,
+ protected void runPlannerTestFile(String testFile, String dbName, TQueryOptions options,
Set<PlannerTestOption> testOptions) {
String fileName = testDir_.resolve(testFile + ".test").toString();
if (options == null) {
diff --git a/fe/src/test/java/org/apache/impala/planner/TpcdsPlannerTest.java b/fe/src/test/java/org/apache/impala/planner/TpcdsPlannerTest.java
index 4c94941..ef8fa40 100644
--- a/fe/src/test/java/org/apache/impala/planner/TpcdsPlannerTest.java
+++ b/fe/src/test/java/org/apache/impala/planner/TpcdsPlannerTest.java
@@ -20,11 +20,13 @@ package org.apache.impala.planner;
import java.nio.file.Paths;
import java.util.Set;
+import org.apache.impala.thrift.TQueryOptions;
import org.junit.BeforeClass;
import org.junit.Test;
import com.google.common.collect.ImmutableSet;
+
/**
* Plans from the TPC-DS qualification queries at scale factor 1. Single node,
* Distributed and Parallel plans are all included. Cardinality and resource checks are
@@ -37,536 +39,540 @@ public class TpcdsPlannerTest extends PlannerTestBase {
PlannerTestOption.INCLUDE_RESOURCE_HEADER, PlannerTestOption.VALIDATE_RESOURCES,
PlannerTestOption.VALIDATE_CARDINALITY);
+ private static TQueryOptions options = new TQueryOptions();
+
@BeforeClass
public static void setUp() throws Exception {
PlannerTestBase.setUp();
Paths.get(outDir_.toString(), "tpcds").toFile().mkdirs();
+ /* Enable minmax overlap filter feature for tpcds Parquet tests. */
+ options.setMinmax_filter_threshold(0.5);
}
@Test
public void testQ1() {
- runPlannerTestFile("tpcds/tpcds-q01", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q01", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ2() {
- runPlannerTestFile("tpcds/tpcds-q02", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q02", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ3() {
- runPlannerTestFile("tpcds/tpcds-q03", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q03", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ4() {
- runPlannerTestFile("tpcds/tpcds-q04", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q04", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ5() {
- runPlannerTestFile("tpcds/tpcds-q05", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q05", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ6() {
- runPlannerTestFile("tpcds/tpcds-q06", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q06", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ7() {
- runPlannerTestFile("tpcds/tpcds-q07", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q07", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ8() {
- runPlannerTestFile("tpcds/tpcds-q08", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q08", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ9() {
- runPlannerTestFile("tpcds/tpcds-q09", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q09", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ10() {
// This is an official variant of q10 that uses a rewrite for lack of support for
// multiple subqueries in disjunctive predicates.
- runPlannerTestFile("tpcds/tpcds-q10a", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q10a", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ11() {
- runPlannerTestFile("tpcds/tpcds-q11", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q11", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ12() {
- runPlannerTestFile("tpcds/tpcds-q12", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q12", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ13() {
- runPlannerTestFile("tpcds/tpcds-q13", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q13", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ14a() {
// First of the two query statements from the official q14.
- runPlannerTestFile("tpcds/tpcds-q14a", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q14a", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ14b() {
// Second of the two query statements from the official q14.
- runPlannerTestFile("tpcds/tpcds-q14b", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q14b", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ15() {
- runPlannerTestFile("tpcds/tpcds-q15", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q15", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ16() {
- runPlannerTestFile("tpcds/tpcds-q16", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q16", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ17() {
- runPlannerTestFile("tpcds/tpcds-q17", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q17", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ18() {
- runPlannerTestFile("tpcds/tpcds-q18", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q18", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ19() {
- runPlannerTestFile("tpcds/tpcds-q19", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q19", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ20() {
- runPlannerTestFile("tpcds/tpcds-q20", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q20", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ21() {
- runPlannerTestFile("tpcds/tpcds-q21", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q21", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ22() {
- runPlannerTestFile("tpcds/tpcds-q22", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q22", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ23a() {
// First of the two query statements from the official q23.
- runPlannerTestFile("tpcds/tpcds-q23a", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q23a", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ23b() {
// Second of the two query statements from the official q23.
- runPlannerTestFile("tpcds/tpcds-q23b", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q23b", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ24a() {
// First of the two query statements from the official q24.
- runPlannerTestFile("tpcds/tpcds-q24a", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q24a", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ24b() {
// Second of the two query statements from the official q24.
- runPlannerTestFile("tpcds/tpcds-q24b", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q24b", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ25() {
- runPlannerTestFile("tpcds/tpcds-q25", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q25", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ26() {
- runPlannerTestFile("tpcds/tpcds-q26", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q26", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ27() {
- runPlannerTestFile("tpcds/tpcds-q27", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q27", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ28() {
- runPlannerTestFile("tpcds/tpcds-q28", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q28", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ29() {
- runPlannerTestFile("tpcds/tpcds-q29", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q29", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ30() {
- runPlannerTestFile("tpcds/tpcds-q30", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q30", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ31() {
- runPlannerTestFile("tpcds/tpcds-q31", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q31", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ32() {
- runPlannerTestFile("tpcds/tpcds-q32", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q32", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ33() {
- runPlannerTestFile("tpcds/tpcds-q33", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q33", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ34() {
- runPlannerTestFile("tpcds/tpcds-q34", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q34", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ35() {
// This is an official variant of q35 that uses a rewrite for lack of support for
// multiple subqueries in disjunctive predicates.
- runPlannerTestFile("tpcds/tpcds-q35a", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q35a", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ36() {
- runPlannerTestFile("tpcds/tpcds-q36", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q36", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ37() {
- runPlannerTestFile("tpcds/tpcds-q37", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q37", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ38() {
- runPlannerTestFile("tpcds/tpcds-q38", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q38", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ39a() {
// First of the two query statements from the official q39.
- runPlannerTestFile("tpcds/tpcds-q39a", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q39a", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ39b() {
// Second of the two query statements from the official q39.
- runPlannerTestFile("tpcds/tpcds-q39b", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q39b", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ40() {
- runPlannerTestFile("tpcds/tpcds-q40", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q40", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ41() {
- runPlannerTestFile("tpcds/tpcds-q41", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q41", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ42() {
- runPlannerTestFile("tpcds/tpcds-q42", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q42", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ43() {
- runPlannerTestFile("tpcds/tpcds-q43", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q43", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ44() {
- runPlannerTestFile("tpcds/tpcds-q44", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q44", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ45() {
- runPlannerTestFile("tpcds/tpcds-q45", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q45", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ46() {
- runPlannerTestFile("tpcds/tpcds-q46", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q46", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ47() {
- runPlannerTestFile("tpcds/tpcds-q47", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q47", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ48() {
- runPlannerTestFile("tpcds/tpcds-q48", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q48", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ49() {
- runPlannerTestFile("tpcds/tpcds-q49", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q49", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ50() {
- runPlannerTestFile("tpcds/tpcds-q50", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q50", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ51() {
- runPlannerTestFile("tpcds/tpcds-q51", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q51", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ52() {
- runPlannerTestFile("tpcds/tpcds-q52", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q52", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ53() {
- runPlannerTestFile("tpcds/tpcds-q53", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q53", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ54() {
- runPlannerTestFile("tpcds/tpcds-q54", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q54", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ55() {
- runPlannerTestFile("tpcds/tpcds-q55", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q55", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ56() {
- runPlannerTestFile("tpcds/tpcds-q56", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q56", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ57() {
- runPlannerTestFile("tpcds/tpcds-q57", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q57", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ58() {
- runPlannerTestFile("tpcds/tpcds-q58", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q58", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ59() {
- runPlannerTestFile("tpcds/tpcds-q59", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q59", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ60() {
- runPlannerTestFile("tpcds/tpcds-q60", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q60", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ61() {
- runPlannerTestFile("tpcds/tpcds-q61", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q61", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ62() {
- runPlannerTestFile("tpcds/tpcds-q62", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q62", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ63() {
- runPlannerTestFile("tpcds/tpcds-q63", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q63", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ64() {
- runPlannerTestFile("tpcds/tpcds-q64", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q64", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ65() {
- runPlannerTestFile("tpcds/tpcds-q65", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q65", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ66() {
- runPlannerTestFile("tpcds/tpcds-q66", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q66", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ67() {
- runPlannerTestFile("tpcds/tpcds-q67", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q67", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ68() {
- runPlannerTestFile("tpcds/tpcds-q68", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q68", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ69() {
- runPlannerTestFile("tpcds/tpcds-q69", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q69", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ70() {
- runPlannerTestFile("tpcds/tpcds-q70", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q70", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ71() {
- runPlannerTestFile("tpcds/tpcds-q71", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q71", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ72() {
- runPlannerTestFile("tpcds/tpcds-q72", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q72", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ73() {
- runPlannerTestFile("tpcds/tpcds-q73", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q73", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ74() {
- runPlannerTestFile("tpcds/tpcds-q74", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q74", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ75() {
- runPlannerTestFile("tpcds/tpcds-q75", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q75", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ76() {
- runPlannerTestFile("tpcds/tpcds-q76", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q76", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ77() {
- runPlannerTestFile("tpcds/tpcds-q77", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q77", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ78() {
- runPlannerTestFile("tpcds/tpcds-q78", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q78", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ79() {
- runPlannerTestFile("tpcds/tpcds-q79", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q79", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ80() {
- runPlannerTestFile("tpcds/tpcds-q80", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q80", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ81() {
- runPlannerTestFile("tpcds/tpcds-q81", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q81", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ82() {
- runPlannerTestFile("tpcds/tpcds-q82", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q82", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ83() {
- runPlannerTestFile("tpcds/tpcds-q83", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q83", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ84() {
- runPlannerTestFile("tpcds/tpcds-q84", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q84", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ85() {
- runPlannerTestFile("tpcds/tpcds-q85", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q85", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ86() {
- runPlannerTestFile("tpcds/tpcds-q86", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q86", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ87() {
- runPlannerTestFile("tpcds/tpcds-q87", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q87", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ88() {
- runPlannerTestFile("tpcds/tpcds-q88", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q88", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ89() {
- runPlannerTestFile("tpcds/tpcds-q89", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q89", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ90() {
- runPlannerTestFile("tpcds/tpcds-q90", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q90", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ91() {
- runPlannerTestFile("tpcds/tpcds-q91", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q91", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ92() {
- runPlannerTestFile("tpcds/tpcds-q92", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q92", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ93() {
- runPlannerTestFile("tpcds/tpcds-q93", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q93", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ94() {
- runPlannerTestFile("tpcds/tpcds-q94", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q94", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ95() {
- runPlannerTestFile("tpcds/tpcds-q95", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q95", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ96() {
- runPlannerTestFile("tpcds/tpcds-q96", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q96", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ97() {
- runPlannerTestFile("tpcds/tpcds-q97", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q97", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ98() {
- runPlannerTestFile("tpcds/tpcds-q98", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q98", "tpcds_parquet", options, testOptions);
}
@Test
public void testQ99() {
- runPlannerTestFile("tpcds/tpcds-q99", "tpcds_parquet", testOptions);
+ runPlannerTestFile("tpcds/tpcds-q99", "tpcds_parquet", options, testOptions);
}
}
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/disable-runtime-overlap-filter.test b/testdata/workloads/functional-planner/queries/PlannerTest/disable-runtime-overlap-filter.test
new file mode 100644
index 0000000..89a7e1f
--- /dev/null
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/disable-runtime-overlap-filter.test
@@ -0,0 +1,29 @@
+###################################################
+# Verify that the overlap predicate is not possible
+# when either enabled_runtime_filter_types is bloom
+# or minmax_filter_threshold is 0.0.
+###################################################
+select straight_join count(*) from
+functional_parquet.alltypes a join [SHUFFLE] functional_parquet.alltypes b
+where a.int_col = b.int_col
+---- PLAN
+PLAN-ROOT SINK
+|
+03:AGGREGATE [FINALIZE]
+| output: count(*)
+| row-size=8B cardinality=1
+|
+02:HASH JOIN [INNER JOIN]
+| hash predicates: a.int_col = b.int_col
+| runtime filters: RF000 <- b.int_col
+| row-size=8B cardinality=12.80K
+|
+|--01:SCAN HDFS [functional_parquet.alltypes b]
+| HDFS partitions=24/24 files=24 size=201.15KB
+| row-size=4B cardinality=12.80K
+|
+00:SCAN HDFS [functional_parquet.alltypes a]
+ HDFS partitions=24/24 files=24 size=201.15KB
+ runtime filters: RF000 -> a.int_col
+ row-size=4B cardinality=12.80K
+====
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/setoperation-rewrite.test b/testdata/workloads/functional-planner/queries/PlannerTest/setoperation-rewrite.test
index 91ae2ae..90ae3cf 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/setoperation-rewrite.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/setoperation-rewrite.test
@@ -89,12 +89,12 @@ PLAN-ROOT SINK
|
|--08:HASH JOIN [LEFT SEMI JOIN]
| | hash predicates: `month` IS NOT DISTINCT FROM `month`, `year` IS NOT DISTINCT FROM `year`, id IS NOT DISTINCT FROM id
-| | runtime filters: RF006 <- `month`, RF007 <- `year`, RF008 <- id
+| | runtime filters: RF003 <- `month`, RF004 <- `year`, RF005 <- id
| | row-size=12B cardinality=2
| |
| |--07:HASH JOIN [LEFT SEMI JOIN]
| | | hash predicates: `month` IS NOT DISTINCT FROM `month`, `year` IS NOT DISTINCT FROM `year`, id IS NOT DISTINCT FROM id
-| | | runtime filters: RF012 <- `month`, RF013 <- `year`, RF014 <- id
+| | | runtime filters: RF006 <- `month`, RF007 <- `year`, RF008 <- id
| | | row-size=12B cardinality=2
| | |
| | |--06:SCAN HDFS [functional.alltypestiny]
@@ -109,7 +109,7 @@ PLAN-ROOT SINK
| | 04:SCAN HDFS [functional.alltypestiny]
| | partition predicates: `year` = 2009, `month` = 2
| | HDFS partitions=1/4 files=1 size=115B
-| | runtime filters: RF012 -> functional.alltypestiny.month, RF013 -> functional.alltypestiny.year, RF014 -> functional.alltypestiny.id
+| | runtime filters: RF006 -> functional.alltypestiny.month, RF007 -> functional.alltypestiny.year, RF008 -> functional.alltypestiny.id
| | row-size=12B cardinality=2
| |
| 03:AGGREGATE [FINALIZE]
@@ -119,7 +119,7 @@ PLAN-ROOT SINK
| 02:SCAN HDFS [functional.alltypestiny]
| partition predicates: `year` = 2009, `month` = 1
| HDFS partitions=1/4 files=1 size=115B
-| runtime filters: RF006 -> functional.alltypestiny.month, RF007 -> functional.alltypestiny.year, RF008 -> functional.alltypestiny.id
+| runtime filters: RF003 -> functional.alltypestiny.month, RF004 -> functional.alltypestiny.year, RF005 -> functional.alltypestiny.id
| row-size=12B cardinality=2
|
01:AGGREGATE [FINALIZE]
@@ -145,14 +145,14 @@ PLAN-ROOT SINK
| |
| 08:HASH JOIN [LEFT SEMI JOIN, PARTITIONED]
| | hash predicates: `month` IS NOT DISTINCT FROM `month`, `year` IS NOT DISTINCT FROM `year`, id IS NOT DISTINCT FROM id
-| | runtime filters: RF006 <- `month`, RF007 <- `year`, RF008 <- id
+| | runtime filters: RF003 <- `month`, RF004 <- `year`, RF005 <- id
| | row-size=12B cardinality=2
| |
| |--17:EXCHANGE [HASH(id,`year`,`month`)]
| | |
| | 07:HASH JOIN [LEFT SEMI JOIN, BROADCAST]
| | | hash predicates: `month` IS NOT DISTINCT FROM `month`, `year` IS NOT DISTINCT FROM `year`, id IS NOT DISTINCT FROM id
-| | | runtime filters: RF012 <- `month`, RF013 <- `year`, RF014 <- id
+| | | runtime filters: RF006 <- `month`, RF007 <- `year`, RF008 <- id
| | | row-size=12B cardinality=2
| | |
| | |--16:EXCHANGE [BROADCAST]
@@ -175,7 +175,7 @@ PLAN-ROOT SINK
| | 04:SCAN HDFS [functional.alltypestiny]
| | partition predicates: `year` = 2009, `month` = 2
| | HDFS partitions=1/4 files=1 size=115B
-| | runtime filters: RF012 -> functional.alltypestiny.month, RF013 -> functional.alltypestiny.year, RF014 -> functional.alltypestiny.id
+| | runtime filters: RF006 -> functional.alltypestiny.month, RF007 -> functional.alltypestiny.year, RF008 -> functional.alltypestiny.id
| | row-size=12B cardinality=2
| |
| 13:AGGREGATE [FINALIZE]
@@ -191,7 +191,7 @@ PLAN-ROOT SINK
| 02:SCAN HDFS [functional.alltypestiny]
| partition predicates: `year` = 2009, `month` = 1
| HDFS partitions=1/4 files=1 size=115B
-| runtime filters: RF006 -> functional.alltypestiny.month, RF007 -> functional.alltypestiny.year, RF008 -> functional.alltypestiny.id
+| runtime filters: RF003 -> functional.alltypestiny.month, RF004 -> functional.alltypestiny.year, RF005 -> functional.alltypestiny.id
| row-size=12B cardinality=2
|
11:AGGREGATE [FINALIZE]
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q01.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q01.test
index d7c846e..4cd5dc3 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q01.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q01.test
@@ -40,7 +40,7 @@ PLAN-ROOT SINK
13:HASH JOIN [LEFT SEMI JOIN]
| hash predicates: sr_store_sk = ctr2.ctr_store_sk
| other join predicates: sum(sr_return_amt) > avg(ctr_total_return) * CAST(1.2 AS DECIMAL(2,1))
-| runtime filters: RF000[bloom] <- ctr2.ctr_store_sk
+| runtime filters: RF000[bloom] <- ctr2.ctr_store_sk, RF001[min_max] <- ctr2.ctr_store_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=5,2,4 row-size=74B cardinality=53.81K
| in pipelines: 05(GETNEXT), 10(OPEN)
@@ -62,7 +62,7 @@ PLAN-ROOT SINK
| 08:HASH JOIN [INNER JOIN]
| | hash predicates: sr_returned_date_sk = d_date_sk
| | fk/pk conjuncts: sr_returned_date_sk = d_date_sk
-| | runtime filters: RF008[bloom] <- d_date_sk
+| | runtime filters: RF008[bloom] <- d_date_sk, RF009[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=6,7 row-size=24B cardinality=53.81K
| | in pipelines: 06(GETNEXT), 07(OPEN)
@@ -82,7 +82,7 @@ PLAN-ROOT SINK
| |
| 06:SCAN HDFS [tpcds_parquet.store_returns]
| HDFS partitions=1/1 files=1 size=15.43MB
-| runtime filters: RF008[bloom] -> sr_returned_date_sk
+| runtime filters: RF009[min_max] -> sr_returned_date_sk, RF008[bloom] -> sr_returned_date_sk
| stored statistics:
| table: rows=287.51K size=15.43MB
| columns: all
@@ -94,7 +94,7 @@ PLAN-ROOT SINK
12:HASH JOIN [INNER JOIN]
| hash predicates: sr_store_sk = s_store_sk
| fk/pk conjuncts: sr_store_sk = s_store_sk
-| runtime filters: RF002[bloom] <- s_store_sk
+| runtime filters: RF002[bloom] <- s_store_sk, RF003[min_max] <- s_store_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=5,2,4 row-size=74B cardinality=53.81K
| in pipelines: 05(GETNEXT), 04(OPEN)
@@ -102,7 +102,7 @@ PLAN-ROOT SINK
|--04:SCAN HDFS [tpcds_parquet.store]
| HDFS partitions=1/1 files=1 size=9.93KB
| predicates: s_state = 'TN'
-| runtime filters: RF000[bloom] -> tpcds_parquet.store.s_store_sk
+| runtime filters: RF001[min_max] -> tpcds_parquet.store.s_store_sk, RF000[bloom] -> tpcds_parquet.store.s_store_sk
| stored statistics:
| table: rows=12 size=9.93KB
| columns: all
@@ -116,7 +116,7 @@ PLAN-ROOT SINK
11:HASH JOIN [INNER JOIN]
| hash predicates: c_customer_sk = sr_customer_sk
| fk/pk conjuncts: none
-| runtime filters: RF004[bloom] <- sr_customer_sk
+| runtime filters: RF004[bloom] <- sr_customer_sk, RF005[min_max] <- sr_customer_sk
| mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
| tuple-ids=5,2 row-size=56B cardinality=53.81K
| in pipelines: 05(GETNEXT), 03(OPEN)
@@ -131,7 +131,7 @@ PLAN-ROOT SINK
| 02:HASH JOIN [INNER JOIN]
| | hash predicates: sr_returned_date_sk = d_date_sk
| | fk/pk conjuncts: sr_returned_date_sk = d_date_sk
-| | runtime filters: RF006[bloom] <- d_date_sk
+| | runtime filters: RF006[bloom] <- d_date_sk, RF007[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=0,1 row-size=24B cardinality=53.81K
| | in pipelines: 00(GETNEXT), 01(OPEN)
@@ -151,7 +151,7 @@ PLAN-ROOT SINK
| |
| 00:SCAN HDFS [tpcds_parquet.store_returns]
| HDFS partitions=1/1 files=1 size=15.43MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.store_returns.sr_store_sk, RF002[bloom] -> tpcds_parquet.store_returns.sr_store_sk, RF006[bloom] -> sr_returned_date_sk
+| runtime filters: RF001[min_max] -> tpcds_parquet.store_returns.sr_store_sk, RF003[min_max] -> tpcds_parquet.store_returns.sr_store_sk, RF007[min_max] -> sr_returned_date_sk, RF000[bloom] -> tpcds_parquet.store_returns.sr_store_sk, RF002[bloom] -> tpcds_parquet.store_returns.sr_store_sk, RF006[bloom] -> sr_returned_date_sk
| stored statistics:
| table: rows=287.51K size=15.43MB
| columns: all
@@ -162,7 +162,7 @@ PLAN-ROOT SINK
|
05:SCAN HDFS [tpcds_parquet.customer]
HDFS partitions=1/1 files=1 size=5.49MB
- runtime filters: RF004[bloom] -> c_customer_sk
+ runtime filters: RF005[min_max] -> c_customer_sk, RF004[bloom] -> c_customer_sk
stored statistics:
table: rows=100.00K size=5.49MB
columns: all
@@ -197,7 +197,7 @@ Per-Host Resources: mem-estimate=43.04MB mem-reservation=11.75MB thread-reservat
13:HASH JOIN [LEFT SEMI JOIN, BROADCAST]
| hash predicates: sr_store_sk = ctr2.ctr_store_sk
| other join predicates: sum(sr_return_amt) > avg(ctr_total_return) * CAST(1.2 AS DECIMAL(2,1))
-| runtime filters: RF000[bloom] <- ctr2.ctr_store_sk
+| runtime filters: RF000[bloom] <- ctr2.ctr_store_sk, RF001[min_max] <- ctr2.ctr_store_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=5,2,4 row-size=74B cardinality=53.81K
| in pipelines: 05(GETNEXT), 24(OPEN)
@@ -254,7 +254,7 @@ Per-Host Resources: mem-estimate=43.04MB mem-reservation=11.75MB thread-reservat
| 08:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: sr_returned_date_sk = d_date_sk
| | fk/pk conjuncts: sr_returned_date_sk = d_date_sk
-| | runtime filters: RF008[bloom] <- d_date_sk
+| | runtime filters: RF008[bloom] <- d_date_sk, RF009[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=6,7 row-size=24B cardinality=53.81K
| | in pipelines: 06(GETNEXT), 07(OPEN)
@@ -281,7 +281,7 @@ Per-Host Resources: mem-estimate=43.04MB mem-reservation=11.75MB thread-reservat
| |
| 06:SCAN HDFS [tpcds_parquet.store_returns, RANDOM]
| HDFS partitions=1/1 files=1 size=15.43MB
-| runtime filters: RF008[bloom] -> sr_returned_date_sk
+| runtime filters: RF009[min_max] -> sr_returned_date_sk, RF008[bloom] -> sr_returned_date_sk
| stored statistics:
| table: rows=287.51K size=15.43MB
| columns: all
@@ -293,7 +293,7 @@ Per-Host Resources: mem-estimate=43.04MB mem-reservation=11.75MB thread-reservat
12:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: sr_store_sk = s_store_sk
| fk/pk conjuncts: sr_store_sk = s_store_sk
-| runtime filters: RF002[bloom] <- s_store_sk
+| runtime filters: RF002[bloom] <- s_store_sk, RF003[min_max] <- s_store_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=5,2,4 row-size=74B cardinality=53.81K
| in pipelines: 05(GETNEXT), 04(OPEN)
@@ -308,7 +308,7 @@ Per-Host Resources: mem-estimate=43.04MB mem-reservation=11.75MB thread-reservat
| 04:SCAN HDFS [tpcds_parquet.store, RANDOM]
| HDFS partitions=1/1 files=1 size=9.93KB
| predicates: s_state = 'TN'
-| runtime filters: RF000[bloom] -> tpcds_parquet.store.s_store_sk
+| runtime filters: RF001[min_max] -> tpcds_parquet.store.s_store_sk, RF000[bloom] -> tpcds_parquet.store.s_store_sk
| stored statistics:
| table: rows=12 size=9.93KB
| columns: all
@@ -322,7 +322,7 @@ Per-Host Resources: mem-estimate=43.04MB mem-reservation=11.75MB thread-reservat
11:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: c_customer_sk = sr_customer_sk
| fk/pk conjuncts: none
-| runtime filters: RF004[bloom] <- sr_customer_sk
+| runtime filters: RF004[bloom] <- sr_customer_sk, RF005[min_max] <- sr_customer_sk
| mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
| tuple-ids=5,2 row-size=56B cardinality=53.81K
| in pipelines: 05(GETNEXT), 17(OPEN)
@@ -358,7 +358,7 @@ Per-Host Resources: mem-estimate=43.04MB mem-reservation=11.75MB thread-reservat
| 02:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: sr_returned_date_sk = d_date_sk
| | fk/pk conjuncts: sr_returned_date_sk = d_date_sk
-| | runtime filters: RF006[bloom] <- d_date_sk
+| | runtime filters: RF006[bloom] <- d_date_sk, RF007[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=0,1 row-size=24B cardinality=53.81K
| | in pipelines: 00(GETNEXT), 01(OPEN)
@@ -385,7 +385,7 @@ Per-Host Resources: mem-estimate=43.04MB mem-reservation=11.75MB thread-reservat
| |
| 00:SCAN HDFS [tpcds_parquet.store_returns, RANDOM]
| HDFS partitions=1/1 files=1 size=15.43MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.store_returns.sr_store_sk, RF002[bloom] -> tpcds_parquet.store_returns.sr_store_sk, RF006[bloom] -> sr_returned_date_sk
+| runtime filters: RF001[min_max] -> tpcds_parquet.store_returns.sr_store_sk, RF003[min_max] -> tpcds_parquet.store_returns.sr_store_sk, RF007[min_max] -> sr_returned_date_sk, RF000[bloom] -> tpcds_parquet.store_returns.sr_store_sk, RF002[bloom] -> tpcds_parquet.store_returns.sr_store_sk, RF006[bloom] -> sr_returned_date_sk
| stored statistics:
| table: rows=287.51K size=15.43MB
| columns: all
@@ -396,7 +396,7 @@ Per-Host Resources: mem-estimate=43.04MB mem-reservation=11.75MB thread-reservat
|
05:SCAN HDFS [tpcds_parquet.customer, RANDOM]
HDFS partitions=1/1 files=1 size=5.49MB
- runtime filters: RF004[bloom] -> c_customer_sk
+ runtime filters: RF005[min_max] -> c_customer_sk, RF004[bloom] -> c_customer_sk
stored statistics:
table: rows=100.00K size=5.49MB
columns: all
@@ -442,7 +442,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=2.00MB thread-reser
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: ctr2.ctr_store_sk
-| | runtime filters: RF000[bloom] <- ctr2.ctr_store_sk
+| | runtime filters: RF000[bloom] <- ctr2.ctr_store_sk, RF001[min_max] <- ctr2.ctr_store_sk
| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| |
| 25:EXCHANGE [BROADCAST]
@@ -508,7 +508,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=2.00MB thread-reser
| | JOIN BUILD
| | | join-table-id=01 plan-id=02 cohort-id=02
| | | build expressions: d_date_sk
-| | | runtime filters: RF008[bloom] <- d_date_sk
+| | | runtime filters: RF008[bloom] <- d_date_sk, RF009[min_max] <- d_date_sk
| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | |
| | 20:EXCHANGE [BROADCAST]
@@ -533,7 +533,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=2.00MB thread-reser
| |
| 06:SCAN HDFS [tpcds_parquet.store_returns, RANDOM]
| HDFS partitions=1/1 files=1 size=15.43MB
-| runtime filters: RF008[bloom] -> sr_returned_date_sk
+| runtime filters: RF009[min_max] -> sr_returned_date_sk, RF008[bloom] -> sr_returned_date_sk
| stored statistics:
| table: rows=287.51K size=15.43MB
| columns: all
@@ -555,7 +555,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=2.00MB thread-reser
| JOIN BUILD
| | join-table-id=02 plan-id=03 cohort-id=01
| | build expressions: s_store_sk
-| | runtime filters: RF002[bloom] <- s_store_sk
+| | runtime filters: RF002[bloom] <- s_store_sk, RF003[min_max] <- s_store_sk
| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| |
| 19:EXCHANGE [BROADCAST]
@@ -569,7 +569,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=2.00MB thread-reser
| 04:SCAN HDFS [tpcds_parquet.store, RANDOM]
| HDFS partitions=1/1 files=1 size=9.93KB
| predicates: s_state = 'TN'
-| runtime filters: RF000[bloom] -> tpcds_parquet.store.s_store_sk
+| runtime filters: RF001[min_max] -> tpcds_parquet.store.s_store_sk, RF000[bloom] -> tpcds_parquet.store.s_store_sk
| stored statistics:
| table: rows=12 size=9.93KB
| columns: all
@@ -593,7 +593,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=2.00MB thread-reser
| JOIN BUILD
| | join-table-id=03 plan-id=04 cohort-id=01
| | build expressions: sr_customer_sk
-| | runtime filters: RF004[bloom] <- sr_customer_sk
+| | runtime filters: RF004[bloom] <- sr_customer_sk, RF005[min_max] <- sr_customer_sk
| | mem-estimate=5.75MB mem-reservation=5.75MB spill-buffer=128.00KB thread-reservation=0
| |
| 18:EXCHANGE [BROADCAST]
@@ -638,7 +638,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=2.00MB thread-reser
| | JOIN BUILD
| | | join-table-id=04 plan-id=05 cohort-id=03
| | | build expressions: d_date_sk
-| | | runtime filters: RF006[bloom] <- d_date_sk
+| | | runtime filters: RF006[bloom] <- d_date_sk, RF007[min_max] <- d_date_sk
| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | |
| | 15:EXCHANGE [BROADCAST]
@@ -663,7 +663,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=2.00MB thread-reser
| |
| 00:SCAN HDFS [tpcds_parquet.store_returns, RANDOM]
| HDFS partitions=1/1 files=1 size=15.43MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.store_returns.sr_store_sk, RF002[bloom] -> tpcds_parquet.store_returns.sr_store_sk, RF006[bloom] -> sr_returned_date_sk
+| runtime filters: RF001[min_max] -> tpcds_parquet.store_returns.sr_store_sk, RF003[min_max] -> tpcds_parquet.store_returns.sr_store_sk, RF007[min_max] -> sr_returned_date_sk, RF000[bloom] -> tpcds_parquet.store_returns.sr_store_sk, RF002[bloom] -> tpcds_parquet.store_returns.sr_store_sk, RF006[bloom] -> sr_returned_date_sk
| stored statistics:
| table: rows=287.51K size=15.43MB
| columns: all
@@ -674,7 +674,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=2.00MB thread-reser
|
05:SCAN HDFS [tpcds_parquet.customer, RANDOM]
HDFS partitions=1/1 files=1 size=5.49MB
- runtime filters: RF004[bloom] -> c_customer_sk
+ runtime filters: RF005[min_max] -> c_customer_sk, RF004[bloom] -> c_customer_sk
stored statistics:
table: rows=100.00K size=5.49MB
columns: all
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q02.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q02.test
index 3f02194..d71433e 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q02.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q02.test
@@ -96,7 +96,7 @@ PLAN-ROOT SINK
16:HASH JOIN [INNER JOIN]
| hash predicates: d_week_seq = d_week_seq - 53
| fk/pk conjuncts: assumed fk/pk
-| runtime filters: RF000[bloom] <- d_week_seq - 53
+| runtime filters: RF000[bloom] <- d_week_seq - 53, RF001[min_max] <- d_week_seq - 53
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=6,8,16,18 row-size=248B cardinality=2.56K
| in pipelines: 05(GETNEXT), 13(OPEN)
@@ -104,7 +104,7 @@ PLAN-ROOT SINK
|--15:HASH JOIN [INNER JOIN]
| | hash predicates: d_week_seq = date_dim.d_week_seq
| | fk/pk conjuncts: none
-| | runtime filters: RF006[bloom] <- date_dim.d_week_seq
+| | runtime filters: RF006[bloom] <- date_dim.d_week_seq, RF007[min_max] <- date_dim.d_week_seq
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=16,18 row-size=124B cardinality=2.56K
| | in pipelines: 13(GETNEXT), 14(OPEN)
@@ -132,14 +132,14 @@ PLAN-ROOT SINK
| 12:HASH JOIN [INNER JOIN]
| | hash predicates: sold_date_sk = d_date_sk
| | fk/pk conjuncts: assumed fk/pk
-| | runtime filters: RF008[bloom] <- d_date_sk
+| | runtime filters: RF008[bloom] <- d_date_sk, RF009[min_max] <- d_date_sk
| | mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
| | tuple-ids=12,15 row-size=35B cardinality=2.16M
| | in pipelines: 09(GETNEXT), 10(GETNEXT), 11(OPEN)
| |
| |--11:SCAN HDFS [tpcds_parquet.date_dim]
| | HDFS partitions=1/1 files=1 size=2.15MB
-| | runtime filters: RF006[bloom] -> tpcds_parquet.date_dim.d_week_seq
+| | runtime filters: RF007[min_max] -> tpcds_parquet.date_dim.d_week_seq, RF006[bloom] -> tpcds_parquet.date_dim.d_week_seq
| | stored statistics:
| | table: rows=73.05K size=2.15MB
| | columns: all
@@ -156,7 +156,7 @@ PLAN-ROOT SINK
| |
| |--10:SCAN HDFS [tpcds_parquet.catalog_sales]
| | HDFS partitions=1/1 files=3 size=96.62MB
-| | runtime filters: RF008[bloom] -> tpcds_parquet.catalog_sales.cs_sold_date_sk
+| | runtime filters: RF009[min_max] -> tpcds_parquet.catalog_sales.cs_sold_date_sk, RF008[bloom] -> tpcds_parquet.catalog_sales.cs_sold_date_sk
| | stored statistics:
| | table: rows=1.44M size=96.62MB
| | columns: all
@@ -167,7 +167,7 @@ PLAN-ROOT SINK
| |
| 09:SCAN HDFS [tpcds_parquet.web_sales]
| HDFS partitions=1/1 files=2 size=45.09MB
-| runtime filters: RF008[bloom] -> tpcds_parquet.web_sales.ws_sold_date_sk
+| runtime filters: RF009[min_max] -> tpcds_parquet.web_sales.ws_sold_date_sk, RF008[bloom] -> tpcds_parquet.web_sales.ws_sold_date_sk
| stored statistics:
| table: rows=719.38K size=45.09MB
| columns: all
@@ -179,7 +179,7 @@ PLAN-ROOT SINK
07:HASH JOIN [INNER JOIN]
| hash predicates: d_week_seq = date_dim.d_week_seq
| fk/pk conjuncts: none
-| runtime filters: RF002[bloom] <- date_dim.d_week_seq
+| runtime filters: RF002[bloom] <- date_dim.d_week_seq, RF003[min_max] <- date_dim.d_week_seq
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=6,8 row-size=124B cardinality=2.56K
| in pipelines: 05(GETNEXT), 06(OPEN)
@@ -187,7 +187,7 @@ PLAN-ROOT SINK
|--06:SCAN HDFS [tpcds_parquet.date_dim]
| HDFS partitions=1/1 files=1 size=2.15MB
| predicates: d_year = CAST(2001 AS INT)
-| runtime filters: RF000[bloom] -> tpcds_parquet.date_dim.d_week_seq
+| runtime filters: RF001[min_max] -> tpcds_parquet.date_dim.d_week_seq, RF000[bloom] -> tpcds_parquet.date_dim.d_week_seq
| stored statistics:
| table: rows=73.05K size=2.15MB
| columns: all
@@ -208,14 +208,14 @@ PLAN-ROOT SINK
04:HASH JOIN [INNER JOIN]
| hash predicates: sold_date_sk = d_date_sk
| fk/pk conjuncts: assumed fk/pk
-| runtime filters: RF004[bloom] <- d_date_sk
+| runtime filters: RF004[bloom] <- d_date_sk, RF005[min_max] <- d_date_sk
| mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
| tuple-ids=2,5 row-size=35B cardinality=2.16M
| in pipelines: 01(GETNEXT), 02(GETNEXT), 03(OPEN)
|
|--03:SCAN HDFS [tpcds_parquet.date_dim]
| HDFS partitions=1/1 files=1 size=2.15MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.date_dim.d_week_seq, RF002[bloom] -> tpcds_parquet.date_dim.d_week_seq
+| runtime filters: RF001[min_max] -> tpcds_parquet.date_dim.d_week_seq, RF003[min_max] -> tpcds_parquet.date_dim.d_week_seq, RF000[bloom] -> tpcds_parquet.date_dim.d_week_seq, RF002[bloom] -> tpcds_parquet.date_dim.d_week_seq
| stored statistics:
| table: rows=73.05K size=2.15MB
| columns: all
@@ -232,7 +232,7 @@ PLAN-ROOT SINK
|
|--02:SCAN HDFS [tpcds_parquet.catalog_sales]
| HDFS partitions=1/1 files=3 size=96.62MB
-| runtime filters: RF004[bloom] -> tpcds_parquet.catalog_sales.cs_sold_date_sk
+| runtime filters: RF005[min_max] -> tpcds_parquet.catalog_sales.cs_sold_date_sk, RF004[bloom] -> tpcds_parquet.catalog_sales.cs_sold_date_sk
| stored statistics:
| table: rows=1.44M size=96.62MB
| columns: all
@@ -243,7 +243,7 @@ PLAN-ROOT SINK
|
01:SCAN HDFS [tpcds_parquet.web_sales]
HDFS partitions=1/1 files=2 size=45.09MB
- runtime filters: RF004[bloom] -> tpcds_parquet.web_sales.ws_sold_date_sk
+ runtime filters: RF005[min_max] -> tpcds_parquet.web_sales.ws_sold_date_sk, RF004[bloom] -> tpcds_parquet.web_sales.ws_sold_date_sk
stored statistics:
table: rows=719.38K size=45.09MB
columns: all
@@ -277,7 +277,7 @@ Per-Host Resources: mem-estimate=21.88MB mem-reservation=13.81MB thread-reservat
16:HASH JOIN [INNER JOIN, PARTITIONED]
| hash predicates: d_week_seq = d_week_seq - 53
| fk/pk conjuncts: assumed fk/pk
-| runtime filters: RF000[bloom] <- d_week_seq - 53
+| runtime filters: RF000[bloom] <- d_week_seq - 53, RF001[min_max] <- d_week_seq - 53
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=6,8,16,18 row-size=248B cardinality=2.56K
| in pipelines: 20(GETNEXT), 24(OPEN)
@@ -292,7 +292,7 @@ Per-Host Resources: mem-estimate=21.88MB mem-reservation=13.81MB thread-reservat
| 15:HASH JOIN [INNER JOIN, PARTITIONED]
| | hash predicates: d_week_seq = date_dim.d_week_seq
| | fk/pk conjuncts: none
-| | runtime filters: RF006[bloom] <- date_dim.d_week_seq
+| | runtime filters: RF006[bloom] <- date_dim.d_week_seq, RF007[min_max] <- date_dim.d_week_seq
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=16,18 row-size=124B cardinality=2.56K
| | in pipelines: 24(GETNEXT), 14(OPEN)
@@ -341,7 +341,7 @@ Per-Host Resources: mem-estimate=21.88MB mem-reservation=13.81MB thread-reservat
| 12:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: sold_date_sk = d_date_sk
| | fk/pk conjuncts: assumed fk/pk
-| | runtime filters: RF008[bloom] <- d_date_sk
+| | runtime filters: RF008[bloom] <- d_date_sk, RF009[min_max] <- d_date_sk
| | mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
| | tuple-ids=12,15 row-size=35B cardinality=2.16M
| | in pipelines: 09(GETNEXT), 10(GETNEXT), 11(OPEN)
@@ -355,7 +355,7 @@ Per-Host Resources: mem-estimate=21.88MB mem-reservation=13.81MB thread-reservat
| | Per-Host Resources: mem-estimate=49.00MB mem-reservation=1.50MB thread-reservation=2 runtime-filters-memory=1.00MB
| | 11:SCAN HDFS [tpcds_parquet.date_dim, RANDOM]
| | HDFS partitions=1/1 files=1 size=2.15MB
-| | runtime filters: RF006[bloom] -> tpcds_parquet.date_dim.d_week_seq
+| | runtime filters: RF007[min_max] -> tpcds_parquet.date_dim.d_week_seq, RF006[bloom] -> tpcds_parquet.date_dim.d_week_seq
| | stored statistics:
| | table: rows=73.05K size=2.15MB
| | columns: all
@@ -372,7 +372,7 @@ Per-Host Resources: mem-estimate=21.88MB mem-reservation=13.81MB thread-reservat
| |
| |--10:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| | HDFS partitions=1/1 files=3 size=96.62MB
-| | runtime filters: RF008[bloom] -> tpcds_parquet.catalog_sales.cs_sold_date_sk
+| | runtime filters: RF009[min_max] -> tpcds_parquet.catalog_sales.cs_sold_date_sk, RF008[bloom] -> tpcds_parquet.catalog_sales.cs_sold_date_sk
| | stored statistics:
| | table: rows=1.44M size=96.62MB
| | columns: all
@@ -383,7 +383,7 @@ Per-Host Resources: mem-estimate=21.88MB mem-reservation=13.81MB thread-reservat
| |
| 09:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| HDFS partitions=1/1 files=2 size=45.09MB
-| runtime filters: RF008[bloom] -> tpcds_parquet.web_sales.ws_sold_date_sk
+| runtime filters: RF009[min_max] -> tpcds_parquet.web_sales.ws_sold_date_sk, RF008[bloom] -> tpcds_parquet.web_sales.ws_sold_date_sk
| stored statistics:
| table: rows=719.38K size=45.09MB
| columns: all
@@ -395,7 +395,7 @@ Per-Host Resources: mem-estimate=21.88MB mem-reservation=13.81MB thread-reservat
07:HASH JOIN [INNER JOIN, PARTITIONED]
| hash predicates: d_week_seq = date_dim.d_week_seq
| fk/pk conjuncts: none
-| runtime filters: RF002[bloom] <- date_dim.d_week_seq
+| runtime filters: RF002[bloom] <- date_dim.d_week_seq, RF003[min_max] <- date_dim.d_week_seq
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=6,8 row-size=124B cardinality=2.56K
| in pipelines: 20(GETNEXT), 06(OPEN)
@@ -410,7 +410,7 @@ Per-Host Resources: mem-estimate=21.88MB mem-reservation=13.81MB thread-reservat
| 06:SCAN HDFS [tpcds_parquet.date_dim, RANDOM]
| HDFS partitions=1/1 files=1 size=2.15MB
| predicates: d_year = CAST(2001 AS INT)
-| runtime filters: RF000[bloom] -> tpcds_parquet.date_dim.d_week_seq
+| runtime filters: RF001[min_max] -> tpcds_parquet.date_dim.d_week_seq, RF000[bloom] -> tpcds_parquet.date_dim.d_week_seq
| stored statistics:
| table: rows=73.05K size=2.15MB
| columns: all
@@ -445,7 +445,7 @@ Per-Host Resources: mem-estimate=111.80MB mem-reservation=10.88MB thread-reserva
04:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: sold_date_sk = d_date_sk
| fk/pk conjuncts: assumed fk/pk
-| runtime filters: RF004[bloom] <- d_date_sk
+| runtime filters: RF004[bloom] <- d_date_sk, RF005[min_max] <- d_date_sk
| mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
| tuple-ids=2,5 row-size=35B cardinality=2.16M
| in pipelines: 01(GETNEXT), 02(GETNEXT), 03(OPEN)
@@ -459,7 +459,7 @@ Per-Host Resources: mem-estimate=111.80MB mem-reservation=10.88MB thread-reserva
| Per-Host Resources: mem-estimate=50.00MB mem-reservation=2.50MB thread-reservation=2 runtime-filters-memory=2.00MB
| 03:SCAN HDFS [tpcds_parquet.date_dim, RANDOM]
| HDFS partitions=1/1 files=1 size=2.15MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.date_dim.d_week_seq, RF002[bloom] -> tpcds_parquet.date_dim.d_week_seq
+| runtime filters: RF001[min_max] -> tpcds_parquet.date_dim.d_week_seq, RF003[min_max] -> tpcds_parquet.date_dim.d_week_seq, RF000[bloom] -> tpcds_parquet.date_dim.d_week_seq, RF002[bloom] -> tpcds_parquet.date_dim.d_week_seq
| stored statistics:
| table: rows=73.05K size=2.15MB
| columns: all
@@ -476,7 +476,7 @@ Per-Host Resources: mem-estimate=111.80MB mem-reservation=10.88MB thread-reserva
|
|--02:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| HDFS partitions=1/1 files=3 size=96.62MB
-| runtime filters: RF004[bloom] -> tpcds_parquet.catalog_sales.cs_sold_date_sk
+| runtime filters: RF005[min_max] -> tpcds_parquet.catalog_sales.cs_sold_date_sk, RF004[bloom] -> tpcds_parquet.catalog_sales.cs_sold_date_sk
| stored statistics:
| table: rows=1.44M size=96.62MB
| columns: all
@@ -487,7 +487,7 @@ Per-Host Resources: mem-estimate=111.80MB mem-reservation=10.88MB thread-reserva
|
01:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
HDFS partitions=1/1 files=2 size=45.09MB
- runtime filters: RF004[bloom] -> tpcds_parquet.web_sales.ws_sold_date_sk
+ runtime filters: RF005[min_max] -> tpcds_parquet.web_sales.ws_sold_date_sk, RF004[bloom] -> tpcds_parquet.web_sales.ws_sold_date_sk
stored statistics:
table: rows=719.38K size=45.09MB
columns: all
@@ -531,7 +531,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=7.94MB thread-reser
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: d_week_seq - 53
-| | runtime filters: RF000[bloom] <- d_week_seq - 53
+| | runtime filters: RF000[bloom] <- d_week_seq - 53, RF001[min_max] <- d_week_seq - 53
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| |
| 26:EXCHANGE [HASH(d_week_seq - 53)]
@@ -554,7 +554,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=7.94MB thread-reser
| | JOIN BUILD
| | | join-table-id=01 plan-id=02 cohort-id=02
| | | build expressions: date_dim.d_week_seq
-| | | runtime filters: RF006[bloom] <- date_dim.d_week_seq
+| | | runtime filters: RF006[bloom] <- date_dim.d_week_seq, RF007[min_max] <- date_dim.d_week_seq
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | |
| | 25:EXCHANGE [HASH(date_dim.d_week_seq)]
@@ -612,7 +612,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=7.94MB thread-reser
| | JOIN BUILD
| | | join-table-id=02 plan-id=03 cohort-id=02
| | | build expressions: d_date_sk
-| | | runtime filters: RF008[bloom] <- d_date_sk
+| | | runtime filters: RF008[bloom] <- d_date_sk, RF009[min_max] <- d_date_sk
| | | mem-estimate=5.75MB mem-reservation=5.75MB spill-buffer=128.00KB thread-reservation=0
| | |
| | 22:EXCHANGE [BROADCAST]
@@ -625,7 +625,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=7.94MB thread-reser
| | Per-Instance Resources: mem-estimate=16.00MB mem-reservation=512.00KB thread-reservation=1
| | 11:SCAN HDFS [tpcds_parquet.date_dim, RANDOM]
| | HDFS partitions=1/1 files=1 size=2.15MB
-| | runtime filters: RF006[bloom] -> tpcds_parquet.date_dim.d_week_seq
+| | runtime filters: RF007[min_max] -> tpcds_parquet.date_dim.d_week_seq, RF006[bloom] -> tpcds_parquet.date_dim.d_week_seq
| | stored statistics:
| | table: rows=73.05K size=2.15MB
| | columns: all
@@ -642,7 +642,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=7.94MB thread-reser
| |
| |--10:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| | HDFS partitions=1/1 files=3 size=96.62MB
-| | runtime filters: RF008[bloom] -> tpcds_parquet.catalog_sales.cs_sold_date_sk
+| | runtime filters: RF009[min_max] -> tpcds_parquet.catalog_sales.cs_sold_date_sk, RF008[bloom] -> tpcds_parquet.catalog_sales.cs_sold_date_sk
| | stored statistics:
| | table: rows=1.44M size=96.62MB
| | columns: all
@@ -653,7 +653,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=7.94MB thread-reser
| |
| 09:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| HDFS partitions=1/1 files=2 size=45.09MB
-| runtime filters: RF008[bloom] -> tpcds_parquet.web_sales.ws_sold_date_sk
+| runtime filters: RF009[min_max] -> tpcds_parquet.web_sales.ws_sold_date_sk, RF008[bloom] -> tpcds_parquet.web_sales.ws_sold_date_sk
| stored statistics:
| table: rows=719.38K size=45.09MB
| columns: all
@@ -675,7 +675,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=7.94MB thread-reser
| JOIN BUILD
| | join-table-id=03 plan-id=04 cohort-id=01
| | build expressions: date_dim.d_week_seq
-| | runtime filters: RF002[bloom] <- date_dim.d_week_seq
+| | runtime filters: RF002[bloom] <- date_dim.d_week_seq, RF003[min_max] <- date_dim.d_week_seq
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| |
| 21:EXCHANGE [HASH(date_dim.d_week_seq)]
@@ -689,7 +689,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=7.94MB thread-reser
| 06:SCAN HDFS [tpcds_parquet.date_dim, RANDOM]
| HDFS partitions=1/1 files=1 size=2.15MB
| predicates: d_year = CAST(2001 AS INT)
-| runtime filters: RF000[bloom] -> tpcds_parquet.date_dim.d_week_seq
+| runtime filters: RF001[min_max] -> tpcds_parquet.date_dim.d_week_seq, RF000[bloom] -> tpcds_parquet.date_dim.d_week_seq
| stored statistics:
| table: rows=73.05K size=2.15MB
| columns: all
@@ -735,7 +735,7 @@ Per-Instance Resources: mem-estimate=58.00MB mem-reservation=7.00MB thread-reser
| JOIN BUILD
| | join-table-id=04 plan-id=05 cohort-id=01
| | build expressions: d_date_sk
-| | runtime filters: RF004[bloom] <- d_date_sk
+| | runtime filters: RF004[bloom] <- d_date_sk, RF005[min_max] <- d_date_sk
| | mem-estimate=5.75MB mem-reservation=5.75MB spill-buffer=128.00KB thread-reservation=0
| |
| 18:EXCHANGE [BROADCAST]
@@ -748,7 +748,7 @@ Per-Instance Resources: mem-estimate=58.00MB mem-reservation=7.00MB thread-reser
| Per-Instance Resources: mem-estimate=16.00MB mem-reservation=512.00KB thread-reservation=1
| 03:SCAN HDFS [tpcds_parquet.date_dim, RANDOM]
| HDFS partitions=1/1 files=1 size=2.15MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.date_dim.d_week_seq, RF002[bloom] -> tpcds_parquet.date_dim.d_week_seq
+| runtime filters: RF001[min_max] -> tpcds_parquet.date_dim.d_week_seq, RF003[min_max] -> tpcds_parquet.date_dim.d_week_seq, RF000[bloom] -> tpcds_parquet.date_dim.d_week_seq, RF002[bloom] -> tpcds_parquet.date_dim.d_week_seq
| stored statistics:
| table: rows=73.05K size=2.15MB
| columns: all
@@ -765,7 +765,7 @@ Per-Instance Resources: mem-estimate=58.00MB mem-reservation=7.00MB thread-reser
|
|--02:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| HDFS partitions=1/1 files=3 size=96.62MB
-| runtime filters: RF004[bloom] -> tpcds_parquet.catalog_sales.cs_sold_date_sk
+| runtime filters: RF005[min_max] -> tpcds_parquet.catalog_sales.cs_sold_date_sk, RF004[bloom] -> tpcds_parquet.catalog_sales.cs_sold_date_sk
| stored statistics:
| table: rows=1.44M size=96.62MB
| columns: all
@@ -776,7 +776,7 @@ Per-Instance Resources: mem-estimate=58.00MB mem-reservation=7.00MB thread-reser
|
01:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
HDFS partitions=1/1 files=2 size=45.09MB
- runtime filters: RF004[bloom] -> tpcds_parquet.web_sales.ws_sold_date_sk
+ runtime filters: RF005[min_max] -> tpcds_parquet.web_sales.ws_sold_date_sk, RF004[bloom] -> tpcds_parquet.web_sales.ws_sold_date_sk
stored statistics:
table: rows=719.38K size=45.09MB
columns: all
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q03.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q03.test
index d79fd72..97ccf42 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q03.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q03.test
@@ -68,7 +68,7 @@ PLAN-ROOT SINK
03:HASH JOIN [INNER JOIN]
| hash predicates: store_sales.ss_item_sk = item.i_item_sk
| fk/pk conjuncts: store_sales.ss_item_sk = item.i_item_sk
-| runtime filters: RF002[bloom] <- item.i_item_sk
+| runtime filters: RF002[bloom] <- item.i_item_sk, RF003[min_max] <- item.i_item_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=1,2 row-size=60B cardinality=3.04K
| in pipelines: 01(GETNEXT), 02(OPEN)
@@ -87,10 +87,10 @@ PLAN-ROOT SINK
| in pipelines: 02(GETNEXT)
|
01:SCAN HDFS [tpcds_parquet.store_sales]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF000[bloom] -> store_sales.ss_sold_date_sk, RF002[bloom] -> store_sales.ss_item_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF003[min_max] -> store_sales.ss_item_sk, RF000[bloom] -> store_sales.ss_sold_date_sk, RF002[bloom] -> store_sales.ss_item_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -180,7 +180,7 @@ Per-Host Resources: mem-estimate=35.95MB mem-reservation=4.94MB thread-reservati
03:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: store_sales.ss_item_sk = item.i_item_sk
| fk/pk conjuncts: store_sales.ss_item_sk = item.i_item_sk
-| runtime filters: RF002[bloom] <- item.i_item_sk
+| runtime filters: RF002[bloom] <- item.i_item_sk, RF003[min_max] <- item.i_item_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=1,2 row-size=60B cardinality=3.04K
| in pipelines: 01(GETNEXT), 02(OPEN)
@@ -206,10 +206,10 @@ Per-Host Resources: mem-estimate=35.95MB mem-reservation=4.94MB thread-reservati
| in pipelines: 02(GETNEXT)
|
01:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF000[bloom] -> store_sales.ss_sold_date_sk, RF002[bloom] -> store_sales.ss_item_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF003[min_max] -> store_sales.ss_item_sk, RF000[bloom] -> store_sales.ss_sold_date_sk, RF002[bloom] -> store_sales.ss_item_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -318,7 +318,7 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=1.00MB thread-reser
| JOIN BUILD
| | join-table-id=01 plan-id=02 cohort-id=01
| | build expressions: item.i_item_sk
-| | runtime filters: RF002[bloom] <- item.i_item_sk
+| | runtime filters: RF002[bloom] <- item.i_item_sk, RF003[min_max] <- item.i_item_sk
| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| |
| 07:EXCHANGE [BROADCAST]
@@ -342,10 +342,10 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=1.00MB thread-reser
| in pipelines: 02(GETNEXT)
|
01:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF000[bloom] -> store_sales.ss_sold_date_sk, RF002[bloom] -> store_sales.ss_item_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF003[min_max] -> store_sales.ss_item_sk, RF000[bloom] -> store_sales.ss_sold_date_sk, RF002[bloom] -> store_sales.ss_item_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q04.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q04.test
index b5aa2c3..0be2431 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q04.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q04.test
@@ -137,7 +137,7 @@ PLAN-ROOT SINK
| hash predicates: customer_id = customer_id
| fk/pk conjuncts: assumed fk/pk
| other predicates: CASE WHEN year_total > CAST(0 AS DECIMAL(3,0)) THEN year_total / year_total ELSE NULL END > CASE WHEN year_total > CAST(0 AS DECIMAL(3,0)) THEN year_total / year_total ELSE NULL END
-| runtime filters: RF000[bloom] <- customer_id
+| runtime filters: RF000[bloom] <- customer_id, RF001[min_max] <- customer_id
| mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| tuple-ids=26,12,40,54,68,82 row-size=313B cardinality=589.03K
| in pipelines: 13(GETNEXT), 41(OPEN)
@@ -157,6 +157,7 @@ PLAN-ROOT SINK
| 40:HASH JOIN [INNER JOIN]
| | hash predicates: c_customer_sk = ws_bill_customer_sk
| | fk/pk conjuncts: c_customer_sk = ws_bill_customer_sk
+| | runtime filters: RF031[min_max] <- ws_bill_customer_sk
| | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| | tuple-ids=78,79,80 row-size=185B cardinality=148.00K
| | in pipelines: 36(GETNEXT), 37(OPEN)
@@ -164,7 +165,7 @@ PLAN-ROOT SINK
| |--39:HASH JOIN [INNER JOIN]
| | | hash predicates: ws_sold_date_sk = d_date_sk
| | | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| | | runtime filters: RF032[bloom] <- d_date_sk
+| | | runtime filters: RF032[bloom] <- d_date_sk, RF033[min_max] <- d_date_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=79,80 row-size=32B cardinality=148.00K
| | | in pipelines: 37(GETNEXT), 38(OPEN)
@@ -184,7 +185,7 @@ PLAN-ROOT SINK
| | |
| | 37:SCAN HDFS [tpcds_parquet.web_sales]
| | HDFS partitions=1/1 files=2 size=45.09MB
-| | runtime filters: RF032[bloom] -> ws_sold_date_sk
+| | runtime filters: RF033[min_max] -> ws_sold_date_sk, RF032[bloom] -> ws_sold_date_sk
| | stored statistics:
| | table: rows=719.38K size=45.09MB
| | columns: all
@@ -195,6 +196,7 @@ PLAN-ROOT SINK
| |
| 36:SCAN HDFS [tpcds_parquet.customer]
| HDFS partitions=1/1 files=1 size=5.49MB
+| runtime filters: RF031[min_max] -> c_customer_sk
| stored statistics:
| table: rows=100.00K size=5.49MB
| columns: all
@@ -206,7 +208,7 @@ PLAN-ROOT SINK
45:HASH JOIN [INNER JOIN]
| hash predicates: customer_id = customer_id
| fk/pk conjuncts: assumed fk/pk
-| runtime filters: RF002[bloom] <- customer_id
+| runtime filters: RF002[bloom] <- customer_id, RF003[min_max] <- customer_id
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=26,12,40,54,68 row-size=269B cardinality=589.03K
| in pipelines: 13(GETNEXT), 34(OPEN)
@@ -227,6 +229,7 @@ PLAN-ROOT SINK
| 33:HASH JOIN [INNER JOIN]
| | hash predicates: c_customer_sk = ws_bill_customer_sk
| | fk/pk conjuncts: c_customer_sk = ws_bill_customer_sk
+| | runtime filters: RF027[min_max] <- ws_bill_customer_sk
| | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| | tuple-ids=64,65,66 row-size=185B cardinality=148.00K
| | in pipelines: 29(GETNEXT), 30(OPEN)
@@ -234,7 +237,7 @@ PLAN-ROOT SINK
| |--32:HASH JOIN [INNER JOIN]
| | | hash predicates: ws_sold_date_sk = d_date_sk
| | | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| | | runtime filters: RF028[bloom] <- d_date_sk
+| | | runtime filters: RF028[bloom] <- d_date_sk, RF029[min_max] <- d_date_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=65,66 row-size=32B cardinality=148.00K
| | | in pipelines: 30(GETNEXT), 31(OPEN)
@@ -254,7 +257,7 @@ PLAN-ROOT SINK
| | |
| | 30:SCAN HDFS [tpcds_parquet.web_sales]
| | HDFS partitions=1/1 files=2 size=45.09MB
-| | runtime filters: RF028[bloom] -> ws_sold_date_sk
+| | runtime filters: RF029[min_max] -> ws_sold_date_sk, RF028[bloom] -> ws_sold_date_sk
| | stored statistics:
| | table: rows=719.38K size=45.09MB
| | columns: all
@@ -265,7 +268,7 @@ PLAN-ROOT SINK
| |
| 29:SCAN HDFS [tpcds_parquet.customer]
| HDFS partitions=1/1 files=1 size=5.49MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id
+| runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF027[min_max] -> c_customer_sk, RF000[bloom] -> tpcds_parquet.customer.c_customer_id
| stored statistics:
| table: rows=100.00K size=5.49MB
| columns: all
@@ -278,7 +281,7 @@ PLAN-ROOT SINK
| hash predicates: customer_id = customer_id
| fk/pk conjuncts: assumed fk/pk
| other predicates: CASE WHEN year_total > CAST(0 AS DECIMAL(3,0)) THEN year_total / year_total ELSE NULL END > CASE WHEN year_total > CAST(0 AS DECIMAL(3,0)) THEN year_total / year_total ELSE NULL END
-| runtime filters: RF004[bloom] <- customer_id
+| runtime filters: RF004[bloom] <- customer_id, RF005[min_max] <- customer_id
| mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB thread-reservation=0
| tuple-ids=26,12,40,54 row-size=225B cardinality=589.03K
| in pipelines: 13(GETNEXT), 27(OPEN)
@@ -298,6 +301,7 @@ PLAN-ROOT SINK
| 26:HASH JOIN [INNER JOIN]
| | hash predicates: c_customer_sk = cs_bill_customer_sk
| | fk/pk conjuncts: c_customer_sk = cs_bill_customer_sk
+| | runtime filters: RF023[min_max] <- cs_bill_customer_sk
| | mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB thread-reservation=0
| | tuple-ids=46,47,48 row-size=185B cardinality=294.63K
| | in pipelines: 22(GETNEXT), 23(OPEN)
@@ -305,7 +309,7 @@ PLAN-ROOT SINK
| |--25:HASH JOIN [INNER JOIN]
| | | hash predicates: cs_sold_date_sk = d_date_sk
| | | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
-| | | runtime filters: RF024[bloom] <- d_date_sk
+| | | runtime filters: RF024[bloom] <- d_date_sk, RF025[min_max] <- d_date_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=47,48 row-size=32B cardinality=294.63K
| | | in pipelines: 23(GETNEXT), 24(OPEN)
@@ -325,7 +329,7 @@ PLAN-ROOT SINK
| | |
| | 23:SCAN HDFS [tpcds_parquet.catalog_sales]
| | HDFS partitions=1/1 files=3 size=96.62MB
-| | runtime filters: RF024[bloom] -> cs_sold_date_sk
+| | runtime filters: RF025[min_max] -> cs_sold_date_sk, RF024[bloom] -> cs_sold_date_sk
| | stored statistics:
| | table: rows=1.44M size=96.62MB
| | columns: all
@@ -336,7 +340,7 @@ PLAN-ROOT SINK
| |
| 22:SCAN HDFS [tpcds_parquet.customer]
| HDFS partitions=1/1 files=1 size=5.49MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id
+| runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF023[min_max] -> c_customer_sk, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id
| stored statistics:
| table: rows=100.00K size=5.49MB
| columns: all
@@ -348,7 +352,7 @@ PLAN-ROOT SINK
43:HASH JOIN [INNER JOIN]
| hash predicates: customer_id = customer_id
| fk/pk conjuncts: assumed fk/pk
-| runtime filters: RF006[bloom] <- customer_id
+| runtime filters: RF006[bloom] <- customer_id, RF007[min_max] <- customer_id
| mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
| tuple-ids=26,12,40 row-size=181B cardinality=589.03K
| in pipelines: 13(GETNEXT), 20(OPEN)
@@ -369,6 +373,7 @@ PLAN-ROOT SINK
| 19:HASH JOIN [INNER JOIN]
| | hash predicates: c_customer_sk = cs_bill_customer_sk
| | fk/pk conjuncts: c_customer_sk = cs_bill_customer_sk
+| | runtime filters: RF019[min_max] <- cs_bill_customer_sk
| | mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB thread-reservation=0
| | tuple-ids=32,33,34 row-size=185B cardinality=294.63K
| | in pipelines: 15(GETNEXT), 16(OPEN)
@@ -376,7 +381,7 @@ PLAN-ROOT SINK
| |--18:HASH JOIN [INNER JOIN]
| | | hash predicates: cs_sold_date_sk = d_date_sk
| | | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
-| | | runtime filters: RF020[bloom] <- d_date_sk
+| | | runtime filters: RF020[bloom] <- d_date_sk, RF021[min_max] <- d_date_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=33,34 row-size=32B cardinality=294.63K
| | | in pipelines: 16(GETNEXT), 17(OPEN)
@@ -396,7 +401,7 @@ PLAN-ROOT SINK
| | |
| | 16:SCAN HDFS [tpcds_parquet.catalog_sales]
| | HDFS partitions=1/1 files=3 size=96.62MB
-| | runtime filters: RF020[bloom] -> cs_sold_date_sk
+| | runtime filters: RF021[min_max] -> cs_sold_date_sk, RF020[bloom] -> cs_sold_date_sk
| | stored statistics:
| | table: rows=1.44M size=96.62MB
| | columns: all
@@ -407,7 +412,7 @@ PLAN-ROOT SINK
| |
| 15:SCAN HDFS [tpcds_parquet.customer]
| HDFS partitions=1/1 files=1 size=5.49MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id
+| runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF005[min_max] -> tpcds_parquet.customer.c_customer_id, RF019[min_max] -> c_customer_sk, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id
| stored statistics:
| table: rows=100.00K size=5.49MB
| columns: all
@@ -419,6 +424,7 @@ PLAN-ROOT SINK
42:HASH JOIN [INNER JOIN]
| hash predicates: customer_id = customer_id
| fk/pk conjuncts: assumed fk/pk
+| runtime filters: RF009[min_max] <- customer_id
| mem-estimate=4.75MB mem-reservation=4.75MB spill-buffer=256.00KB thread-reservation=0
| tuple-ids=26,12 row-size=137B cardinality=589.03K
| in pipelines: 13(GETNEXT), 06(OPEN)
@@ -439,13 +445,14 @@ PLAN-ROOT SINK
| 05:HASH JOIN [INNER JOIN]
| | hash predicates: ss_customer_sk = c_customer_sk
| | fk/pk conjuncts: ss_customer_sk = c_customer_sk
+| | runtime filters: RF015[min_max] <- c_customer_sk
| | mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB thread-reservation=0
| | tuple-ids=1,2,0 row-size=185B cardinality=589.03K
| | in pipelines: 02(GETNEXT), 01(OPEN)
| |
| |--01:SCAN HDFS [tpcds_parquet.customer]
| | HDFS partitions=1/1 files=1 size=5.49MB
-| | runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id, RF006[bloom] -> tpcds_parquet.customer.c_customer_id
+| | runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF005[min_max] -> tpcds_parquet.customer.c_customer_id, RF007[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id, RF006[bloom] -> tpcds_parquet.customer.c_customer_id
| | stored statistics:
| | table: rows=100.00K size=5.49MB
| | columns: all
@@ -476,10 +483,10 @@ PLAN-ROOT SINK
| | in pipelines: 03(GETNEXT)
| |
| 02:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
-| runtime filters: RF016[bloom] -> ss_sold_date_sk
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
+| runtime filters: RF015[min_max] -> ss_customer_sk, RF016[bloom] -> ss_sold_date_sk
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -502,13 +509,14 @@ PLAN-ROOT SINK
12:HASH JOIN [INNER JOIN]
| hash predicates: ss_customer_sk = c_customer_sk
| fk/pk conjuncts: ss_customer_sk = c_customer_sk
+| runtime filters: RF011[min_max] <- c_customer_sk
| mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB thread-reservation=0
| tuple-ids=15,16,14 row-size=185B cardinality=589.03K
| in pipelines: 09(GETNEXT), 08(OPEN)
|
|--08:SCAN HDFS [tpcds_parquet.customer]
| HDFS partitions=1/1 files=1 size=5.49MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id, RF006[bloom] -> tpcds_parquet.customer.c_customer_id
+| runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF005[min_max] -> tpcds_parquet.customer.c_customer_id, RF007[min_max] -> tpcds_parquet.customer.c_customer_id, RF009[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id, RF006[bloom] -> tpcds_parqu [...]
| stored statistics:
| table: rows=100.00K size=5.49MB
| columns: all
@@ -539,10 +547,10 @@ PLAN-ROOT SINK
| in pipelines: 10(GETNEXT)
|
09:SCAN HDFS [tpcds_parquet.store_sales]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF012[bloom] -> ss_sold_date_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF011[min_max] -> ss_customer_sk, RF012[bloom] -> ss_sold_date_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -577,7 +585,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| hash predicates: customer_id = customer_id
| fk/pk conjuncts: assumed fk/pk
| other predicates: CASE WHEN year_total > CAST(0 AS DECIMAL(3,0)) THEN year_total / year_total ELSE NULL END > CASE WHEN year_total > CAST(0 AS DECIMAL(3,0)) THEN year_total / year_total ELSE NULL END
-| runtime filters: RF000[bloom] <- customer_id
+| runtime filters: RF000[bloom] <- customer_id, RF001[min_max] <- customer_id
| mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| tuple-ids=26,12,40,54,68,82 row-size=313B cardinality=589.03K
| in pipelines: 52(GETNEXT), 81(OPEN)
@@ -618,6 +626,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| 40:HASH JOIN [INNER JOIN, PARTITIONED]
| | hash predicates: ws_bill_customer_sk = c_customer_sk
| | fk/pk conjuncts: ws_bill_customer_sk = c_customer_sk
+| | runtime filters: RF031[min_max] <- c_customer_sk
| | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| | tuple-ids=79,80,78 row-size=185B cardinality=148.00K
| | in pipelines: 37(GETNEXT), 36(OPEN)
@@ -649,7 +658,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| 39:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: ws_sold_date_sk = d_date_sk
| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| | runtime filters: RF032[bloom] <- d_date_sk
+| | runtime filters: RF032[bloom] <- d_date_sk, RF033[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=79,80 row-size=32B cardinality=148.00K
| | in pipelines: 37(GETNEXT), 38(OPEN)
@@ -676,7 +685,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| |
| 37:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| HDFS partitions=1/1 files=2 size=45.09MB
-| runtime filters: RF032[bloom] -> ws_sold_date_sk
+| runtime filters: RF033[min_max] -> ws_sold_date_sk, RF031[min_max] -> ws_bill_customer_sk, RF032[bloom] -> ws_sold_date_sk
| stored statistics:
| table: rows=719.38K size=45.09MB
| columns: all
@@ -688,7 +697,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
45:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: customer_id = customer_id
| fk/pk conjuncts: assumed fk/pk
-| runtime filters: RF002[bloom] <- customer_id
+| runtime filters: RF002[bloom] <- customer_id, RF003[min_max] <- customer_id
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=26,12,40,54,68 row-size=269B cardinality=589.03K
| in pipelines: 52(GETNEXT), 75(OPEN)
@@ -730,6 +739,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| 33:HASH JOIN [INNER JOIN, PARTITIONED]
| | hash predicates: ws_bill_customer_sk = c_customer_sk
| | fk/pk conjuncts: ws_bill_customer_sk = c_customer_sk
+| | runtime filters: RF027[min_max] <- c_customer_sk
| | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| | tuple-ids=65,66,64 row-size=185B cardinality=148.00K
| | in pipelines: 30(GETNEXT), 29(OPEN)
@@ -743,7 +753,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| | Per-Host Resources: mem-estimate=129.00MB mem-reservation=9.00MB thread-reservation=2 runtime-filters-memory=1.00MB
| | 29:SCAN HDFS [tpcds_parquet.customer, RANDOM]
| | HDFS partitions=1/1 files=1 size=5.49MB
-| | runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id
+| | runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id
| | stored statistics:
| | table: rows=100.00K size=5.49MB
| | columns: all
@@ -762,7 +772,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| 32:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: ws_sold_date_sk = d_date_sk
| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| | runtime filters: RF028[bloom] <- d_date_sk
+| | runtime filters: RF028[bloom] <- d_date_sk, RF029[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=65,66 row-size=32B cardinality=148.00K
| | in pipelines: 30(GETNEXT), 31(OPEN)
@@ -789,7 +799,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| |
| 30:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| HDFS partitions=1/1 files=2 size=45.09MB
-| runtime filters: RF028[bloom] -> ws_sold_date_sk
+| runtime filters: RF029[min_max] -> ws_sold_date_sk, RF027[min_max] -> ws_bill_customer_sk, RF028[bloom] -> ws_sold_date_sk
| stored statistics:
| table: rows=719.38K size=45.09MB
| columns: all
@@ -802,7 +812,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| hash predicates: customer_id = customer_id
| fk/pk conjuncts: assumed fk/pk
| other predicates: CASE WHEN year_total > CAST(0 AS DECIMAL(3,0)) THEN year_total / year_total ELSE NULL END > CASE WHEN year_total > CAST(0 AS DECIMAL(3,0)) THEN year_total / year_total ELSE NULL END
-| runtime filters: RF004[bloom] <- customer_id
+| runtime filters: RF004[bloom] <- customer_id, RF005[min_max] <- customer_id
| mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB thread-reservation=0
| tuple-ids=26,12,40,54 row-size=225B cardinality=589.03K
| in pipelines: 52(GETNEXT), 69(OPEN)
@@ -843,6 +853,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| 26:HASH JOIN [INNER JOIN, PARTITIONED]
| | hash predicates: cs_bill_customer_sk = c_customer_sk
| | fk/pk conjuncts: cs_bill_customer_sk = c_customer_sk
+| | runtime filters: RF023[min_max] <- c_customer_sk
| | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| | tuple-ids=47,48,46 row-size=185B cardinality=294.63K
| | in pipelines: 23(GETNEXT), 22(OPEN)
@@ -856,7 +867,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| | Per-Host Resources: mem-estimate=130.00MB mem-reservation=10.00MB thread-reservation=2 runtime-filters-memory=2.00MB
| | 22:SCAN HDFS [tpcds_parquet.customer, RANDOM]
| | HDFS partitions=1/1 files=1 size=5.49MB
-| | runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id
+| | runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id
| | stored statistics:
| | table: rows=100.00K size=5.49MB
| | columns: all
@@ -875,7 +886,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| 25:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: cs_sold_date_sk = d_date_sk
| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
-| | runtime filters: RF024[bloom] <- d_date_sk
+| | runtime filters: RF024[bloom] <- d_date_sk, RF025[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=47,48 row-size=32B cardinality=294.63K
| | in pipelines: 23(GETNEXT), 24(OPEN)
@@ -902,7 +913,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| |
| 23:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| HDFS partitions=1/1 files=3 size=96.62MB
-| runtime filters: RF024[bloom] -> cs_sold_date_sk
+| runtime filters: RF025[min_max] -> cs_sold_date_sk, RF023[min_max] -> cs_bill_customer_sk, RF024[bloom] -> cs_sold_date_sk
| stored statistics:
| table: rows=1.44M size=96.62MB
| columns: all
@@ -914,7 +925,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
43:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: customer_id = customer_id
| fk/pk conjuncts: assumed fk/pk
-| runtime filters: RF006[bloom] <- customer_id
+| runtime filters: RF006[bloom] <- customer_id, RF007[min_max] <- customer_id
| mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
| tuple-ids=26,12,40 row-size=181B cardinality=589.03K
| in pipelines: 52(GETNEXT), 63(OPEN)
@@ -956,6 +967,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| 19:HASH JOIN [INNER JOIN, PARTITIONED]
| | hash predicates: cs_bill_customer_sk = c_customer_sk
| | fk/pk conjuncts: cs_bill_customer_sk = c_customer_sk
+| | runtime filters: RF019[min_max] <- c_customer_sk
| | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| | tuple-ids=33,34,32 row-size=185B cardinality=294.63K
| | in pipelines: 16(GETNEXT), 15(OPEN)
@@ -969,7 +981,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| | Per-Host Resources: mem-estimate=131.00MB mem-reservation=11.00MB thread-reservation=2 runtime-filters-memory=3.00MB
| | 15:SCAN HDFS [tpcds_parquet.customer, RANDOM]
| | HDFS partitions=1/1 files=1 size=5.49MB
-| | runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id
+| | runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF005[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id
| | stored statistics:
| | table: rows=100.00K size=5.49MB
| | columns: all
@@ -988,7 +1000,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| 18:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: cs_sold_date_sk = d_date_sk
| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
-| | runtime filters: RF020[bloom] <- d_date_sk
+| | runtime filters: RF020[bloom] <- d_date_sk, RF021[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=33,34 row-size=32B cardinality=294.63K
| | in pipelines: 16(GETNEXT), 17(OPEN)
@@ -1015,7 +1027,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| |
| 16:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| HDFS partitions=1/1 files=3 size=96.62MB
-| runtime filters: RF020[bloom] -> cs_sold_date_sk
+| runtime filters: RF021[min_max] -> cs_sold_date_sk, RF019[min_max] -> cs_bill_customer_sk, RF020[bloom] -> cs_sold_date_sk
| stored statistics:
| table: rows=1.44M size=96.62MB
| columns: all
@@ -1027,6 +1039,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
42:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: customer_id = customer_id
| fk/pk conjuncts: assumed fk/pk
+| runtime filters: RF009[min_max] <- customer_id
| mem-estimate=4.75MB mem-reservation=4.75MB spill-buffer=256.00KB thread-reservation=0
| tuple-ids=26,12 row-size=137B cardinality=589.03K
| in pipelines: 52(GETNEXT), 57(OPEN)
@@ -1068,6 +1081,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| 05:HASH JOIN [INNER JOIN, PARTITIONED]
| | hash predicates: ss_customer_sk = c_customer_sk
| | fk/pk conjuncts: ss_customer_sk = c_customer_sk
+| | runtime filters: RF015[min_max] <- c_customer_sk
| | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| | tuple-ids=1,2,0 row-size=185B cardinality=589.03K
| | in pipelines: 02(GETNEXT), 01(OPEN)
@@ -1081,7 +1095,7 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| | Per-Host Resources: mem-estimate=132.00MB mem-reservation=12.00MB thread-reservation=2 runtime-filters-memory=4.00MB
| | 01:SCAN HDFS [tpcds_parquet.customer, RANDOM]
| | HDFS partitions=1/1 files=1 size=5.49MB
-| | runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id, RF006[bloom] -> tpcds_parquet.customer.c_customer_id
+| | runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF005[min_max] -> tpcds_parquet.customer.c_customer_id, RF007[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id, RF006[bloom] -> tpcds_parquet.customer.c_customer_id
| | stored statistics:
| | table: rows=100.00K size=5.49MB
| | columns: all
@@ -1126,10 +1140,10 @@ Per-Host Resources: mem-estimate=122.97MB mem-reservation=73.06MB thread-reserva
| | in pipelines: 03(GETNEXT)
| |
| 02:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
-| runtime filters: RF016[bloom] -> ss_sold_date_sk
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
+| runtime filters: RF015[min_max] -> ss_customer_sk, RF016[bloom] -> ss_sold_date_sk
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1166,6 +1180,7 @@ Per-Host Resources: mem-estimate=77.01MB mem-reservation=42.50MB thread-reservat
12:HASH JOIN [INNER JOIN, PARTITIONED]
| hash predicates: ss_customer_sk = c_customer_sk
| fk/pk conjuncts: ss_customer_sk = c_customer_sk
+| runtime filters: RF011[min_max] <- c_customer_sk
| mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| tuple-ids=15,16,14 row-size=185B cardinality=589.03K
| in pipelines: 09(GETNEXT), 08(OPEN)
@@ -1179,7 +1194,7 @@ Per-Host Resources: mem-estimate=77.01MB mem-reservation=42.50MB thread-reservat
| Per-Host Resources: mem-estimate=132.00MB mem-reservation=12.00MB thread-reservation=2 runtime-filters-memory=4.00MB
| 08:SCAN HDFS [tpcds_parquet.customer, RANDOM]
| HDFS partitions=1/1 files=1 size=5.49MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id, RF006[bloom] -> tpcds_parquet.customer.c_customer_id
+| runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF005[min_max] -> tpcds_parquet.customer.c_customer_id, RF007[min_max] -> tpcds_parquet.customer.c_customer_id, RF009[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id, RF006[bloom] -> tpcds_parqu [...]
| stored statistics:
| table: rows=100.00K size=5.49MB
| columns: all
@@ -1224,10 +1239,10 @@ Per-Host Resources: mem-estimate=82.95MB mem-reservation=6.94MB thread-reservati
| in pipelines: 10(GETNEXT)
|
09:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF012[bloom] -> ss_sold_date_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF011[min_max] -> ss_customer_sk, RF012[bloom] -> ss_sold_date_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1272,7 +1287,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: customer_id
-| | runtime filters: RF000[bloom] <- customer_id
+| | runtime filters: RF000[bloom] <- customer_id, RF001[min_max] <- customer_id
| | mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=512.00KB thread-reservation=0
| |
| 82:EXCHANGE [BROADCAST]
@@ -1321,6 +1336,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | JOIN BUILD
| | | join-table-id=01 plan-id=02 cohort-id=02
| | | build expressions: c_customer_sk
+| | | runtime filters: RF031[min_max] <- c_customer_sk
| | | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| | |
| | 79:EXCHANGE [HASH(c_customer_sk)]
@@ -1361,7 +1377,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | JOIN BUILD
| | | join-table-id=02 plan-id=03 cohort-id=02
| | | build expressions: d_date_sk
-| | | runtime filters: RF032[bloom] <- d_date_sk
+| | | runtime filters: RF032[bloom] <- d_date_sk, RF033[min_max] <- d_date_sk
| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | |
| | 77:EXCHANGE [BROADCAST]
@@ -1386,7 +1402,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| |
| 37:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| HDFS partitions=1/1 files=2 size=45.09MB
-| runtime filters: RF032[bloom] -> ws_sold_date_sk
+| runtime filters: RF033[min_max] -> ws_sold_date_sk, RF031[min_max] -> ws_bill_customer_sk, RF032[bloom] -> ws_sold_date_sk
| stored statistics:
| table: rows=719.38K size=45.09MB
| columns: all
@@ -1408,7 +1424,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| JOIN BUILD
| | join-table-id=03 plan-id=04 cohort-id=01
| | build expressions: customer_id
-| | runtime filters: RF002[bloom] <- customer_id
+| | runtime filters: RF002[bloom] <- customer_id, RF003[min_max] <- customer_id
| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| |
| 76:EXCHANGE [BROADCAST]
@@ -1458,6 +1474,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | JOIN BUILD
| | | join-table-id=04 plan-id=05 cohort-id=03
| | | build expressions: c_customer_sk
+| | | runtime filters: RF027[min_max] <- c_customer_sk
| | | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| | |
| | 73:EXCHANGE [HASH(c_customer_sk)]
@@ -1470,7 +1487,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | Per-Instance Resources: mem-estimate=16.00MB mem-reservation=8.00MB thread-reservation=1
| | 29:SCAN HDFS [tpcds_parquet.customer, RANDOM]
| | HDFS partitions=1/1 files=1 size=5.49MB
-| | runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id
+| | runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id
| | stored statistics:
| | table: rows=100.00K size=5.49MB
| | columns: all
@@ -1500,7 +1517,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | JOIN BUILD
| | | join-table-id=05 plan-id=06 cohort-id=03
| | | build expressions: d_date_sk
-| | | runtime filters: RF028[bloom] <- d_date_sk
+| | | runtime filters: RF028[bloom] <- d_date_sk, RF029[min_max] <- d_date_sk
| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | |
| | 71:EXCHANGE [BROADCAST]
@@ -1525,7 +1542,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| |
| 30:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| HDFS partitions=1/1 files=2 size=45.09MB
-| runtime filters: RF028[bloom] -> ws_sold_date_sk
+| runtime filters: RF029[min_max] -> ws_sold_date_sk, RF027[min_max] -> ws_bill_customer_sk, RF028[bloom] -> ws_sold_date_sk
| stored statistics:
| table: rows=719.38K size=45.09MB
| columns: all
@@ -1548,7 +1565,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| JOIN BUILD
| | join-table-id=06 plan-id=07 cohort-id=01
| | build expressions: customer_id
-| | runtime filters: RF004[bloom] <- customer_id
+| | runtime filters: RF004[bloom] <- customer_id, RF005[min_max] <- customer_id
| | mem-estimate=34.00MB mem-reservation=34.00MB spill-buffer=1.00MB thread-reservation=0
| |
| 70:EXCHANGE [BROADCAST]
@@ -1597,6 +1614,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | JOIN BUILD
| | | join-table-id=07 plan-id=08 cohort-id=04
| | | build expressions: c_customer_sk
+| | | runtime filters: RF023[min_max] <- c_customer_sk
| | | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| | |
| | 67:EXCHANGE [HASH(c_customer_sk)]
@@ -1609,7 +1627,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | Per-Instance Resources: mem-estimate=16.00MB mem-reservation=8.00MB thread-reservation=1
| | 22:SCAN HDFS [tpcds_parquet.customer, RANDOM]
| | HDFS partitions=1/1 files=1 size=5.49MB
-| | runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id
+| | runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id
| | stored statistics:
| | table: rows=100.00K size=5.49MB
| | columns: all
@@ -1639,7 +1657,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | JOIN BUILD
| | | join-table-id=08 plan-id=09 cohort-id=04
| | | build expressions: d_date_sk
-| | | runtime filters: RF024[bloom] <- d_date_sk
+| | | runtime filters: RF024[bloom] <- d_date_sk, RF025[min_max] <- d_date_sk
| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | |
| | 65:EXCHANGE [BROADCAST]
@@ -1664,7 +1682,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| |
| 23:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| HDFS partitions=1/1 files=3 size=96.62MB
-| runtime filters: RF024[bloom] -> cs_sold_date_sk
+| runtime filters: RF025[min_max] -> cs_sold_date_sk, RF023[min_max] -> cs_bill_customer_sk, RF024[bloom] -> cs_sold_date_sk
| stored statistics:
| table: rows=1.44M size=96.62MB
| columns: all
@@ -1686,7 +1704,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| JOIN BUILD
| | join-table-id=09 plan-id=10 cohort-id=01
| | build expressions: customer_id
-| | runtime filters: RF006[bloom] <- customer_id
+| | runtime filters: RF006[bloom] <- customer_id, RF007[min_max] <- customer_id
| | mem-estimate=5.75MB mem-reservation=5.75MB spill-buffer=128.00KB thread-reservation=0
| |
| 64:EXCHANGE [BROADCAST]
@@ -1736,6 +1754,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | JOIN BUILD
| | | join-table-id=10 plan-id=11 cohort-id=05
| | | build expressions: c_customer_sk
+| | | runtime filters: RF019[min_max] <- c_customer_sk
| | | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| | |
| | 61:EXCHANGE [HASH(c_customer_sk)]
@@ -1748,7 +1767,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | Per-Instance Resources: mem-estimate=16.00MB mem-reservation=8.00MB thread-reservation=1
| | 15:SCAN HDFS [tpcds_parquet.customer, RANDOM]
| | HDFS partitions=1/1 files=1 size=5.49MB
-| | runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id
+| | runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF005[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id
| | stored statistics:
| | table: rows=100.00K size=5.49MB
| | columns: all
@@ -1778,7 +1797,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | JOIN BUILD
| | | join-table-id=11 plan-id=12 cohort-id=05
| | | build expressions: d_date_sk
-| | | runtime filters: RF020[bloom] <- d_date_sk
+| | | runtime filters: RF020[bloom] <- d_date_sk, RF021[min_max] <- d_date_sk
| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | |
| | 59:EXCHANGE [BROADCAST]
@@ -1803,7 +1822,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| |
| 16:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| HDFS partitions=1/1 files=3 size=96.62MB
-| runtime filters: RF020[bloom] -> cs_sold_date_sk
+| runtime filters: RF021[min_max] -> cs_sold_date_sk, RF019[min_max] -> cs_bill_customer_sk, RF020[bloom] -> cs_sold_date_sk
| stored statistics:
| table: rows=1.44M size=96.62MB
| columns: all
@@ -1825,6 +1844,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| JOIN BUILD
| | join-table-id=12 plan-id=13 cohort-id=01
| | build expressions: customer_id
+| | runtime filters: RF009[min_max] <- customer_id
| | mem-estimate=9.50MB mem-reservation=9.50MB spill-buffer=256.00KB thread-reservation=0
| |
| 58:EXCHANGE [BROADCAST]
@@ -1874,6 +1894,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | JOIN BUILD
| | | join-table-id=13 plan-id=14 cohort-id=06
| | | build expressions: c_customer_sk
+| | | runtime filters: RF015[min_max] <- c_customer_sk
| | | mem-estimate=4.75MB mem-reservation=4.75MB spill-buffer=256.00KB thread-reservation=0
| | |
| | 55:EXCHANGE [HASH(c_customer_sk)]
@@ -1886,7 +1907,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | Per-Instance Resources: mem-estimate=16.00MB mem-reservation=8.00MB thread-reservation=1
| | 01:SCAN HDFS [tpcds_parquet.customer, RANDOM]
| | HDFS partitions=1/1 files=1 size=5.49MB
-| | runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id, RF006[bloom] -> tpcds_parquet.customer.c_customer_id
+| | runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF005[min_max] -> tpcds_parquet.customer.c_customer_id, RF007[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id, RF006[bloom] -> tpcds_parquet.customer.c_customer_id
| | stored statistics:
| | table: rows=100.00K size=5.49MB
| | columns: all
@@ -1940,10 +1961,10 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | in pipelines: 03(GETNEXT)
| |
| 02:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
-| runtime filters: RF016[bloom] -> ss_sold_date_sk
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
+| runtime filters: RF015[min_max] -> ss_customer_sk, RF016[bloom] -> ss_sold_date_sk
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1990,6 +2011,7 @@ Per-Instance Resources: mem-estimate=40.23MB mem-reservation=34.00MB thread-rese
| JOIN BUILD
| | join-table-id=15 plan-id=16 cohort-id=01
| | build expressions: c_customer_sk
+| | runtime filters: RF011[min_max] <- c_customer_sk
| | mem-estimate=4.75MB mem-reservation=4.75MB spill-buffer=256.00KB thread-reservation=0
| |
| 50:EXCHANGE [HASH(c_customer_sk)]
@@ -2002,7 +2024,7 @@ Per-Instance Resources: mem-estimate=40.23MB mem-reservation=34.00MB thread-rese
| Per-Instance Resources: mem-estimate=16.00MB mem-reservation=8.00MB thread-reservation=1
| 08:SCAN HDFS [tpcds_parquet.customer, RANDOM]
| HDFS partitions=1/1 files=1 size=5.49MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id, RF006[bloom] -> tpcds_parquet.customer.c_customer_id
+| runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF005[min_max] -> tpcds_parquet.customer.c_customer_id, RF007[min_max] -> tpcds_parquet.customer.c_customer_id, RF009[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id, RF006[bloom] -> tpcds_parqu [...]
| stored statistics:
| table: rows=100.00K size=5.49MB
| columns: all
@@ -2056,10 +2078,10 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=4.00MB thread-reser
| in pipelines: 10(GETNEXT)
|
09:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF012[bloom] -> ss_sold_date_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF011[min_max] -> ss_customer_sk, RF012[bloom] -> ss_sold_date_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q05.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q05.test
index 4da019e..78f0f4f 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q05.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q05.test
@@ -175,7 +175,7 @@ PLAN-ROOT SINK
| 25:HASH JOIN [INNER JOIN]
| | hash predicates: wsr_web_site_sk = web_site_sk
| | fk/pk conjuncts: assumed fk/pk
-| | runtime filters: RF008[bloom] <- web_site_sk
+| | runtime filters: RF008[bloom] <- web_site_sk, RF009[min_max] <- web_site_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=19,21,22 row-size=82B cardinality=791.15K
| | in pipelines: 18(GETNEXT), 20(GETNEXT), 23(OPEN)
@@ -193,7 +193,7 @@ PLAN-ROOT SINK
| 24:HASH JOIN [INNER JOIN]
| | hash predicates: date_sk = d_date_sk
| | fk/pk conjuncts: assumed fk/pk
-| | runtime filters: RF010[bloom] <- d_date_sk
+| | runtime filters: RF010[bloom] <- d_date_sk, RF011[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=19,21 row-size=50B cardinality=791.15K
| | in pipelines: 18(GETNEXT), 20(GETNEXT), 22(OPEN)
@@ -219,14 +219,14 @@ PLAN-ROOT SINK
| |--21:HASH JOIN [RIGHT OUTER JOIN]
| | | hash predicates: ws_item_sk = wr_item_sk, ws_order_number = wr_order_number
| | | fk/pk conjuncts: ws_item_sk = wr_item_sk, ws_order_number = wr_order_number
-| | | runtime filters: RF012[bloom] <- wr_item_sk, RF013[bloom] <- wr_order_number
+| | | runtime filters: RF012[bloom] <- wr_item_sk, RF013[bloom] <- wr_order_number, RF014[min_max] <- wr_item_sk, RF015[min_max] <- wr_order_number
| | | mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
| | | tuple-ids=18N,17 row-size=48B cardinality=71.76K
| | | in pipelines: 20(GETNEXT), 19(OPEN)
| | |
| | |--19:SCAN HDFS [tpcds_parquet.web_returns]
| | | HDFS partitions=1/1 files=1 size=5.66MB
-| | | runtime filters: RF010[bloom] -> tpcds_parquet.web_returns.wr_returned_date_sk
+| | | runtime filters: RF011[min_max] -> tpcds_parquet.web_returns.wr_returned_date_sk, RF010[bloom] -> tpcds_parquet.web_returns.wr_returned_date_sk
| | | stored statistics:
| | | table: rows=71.76K size=5.66MB
| | | columns: all
@@ -237,7 +237,7 @@ PLAN-ROOT SINK
| | |
| | 20:SCAN HDFS [tpcds_parquet.web_sales]
| | HDFS partitions=1/1 files=2 size=45.09MB
-| | runtime filters: RF008[bloom] -> tpcds_parquet.web_sales.ws_web_site_sk, RF012[bloom] -> ws_item_sk, RF013[bloom] -> ws_order_number
+| | runtime filters: RF014[min_max] -> ws_item_sk, RF015[min_max] -> ws_order_number, RF009[min_max] -> tpcds_parquet.web_sales.ws_web_site_sk, RF012[bloom] -> ws_item_sk, RF013[bloom] -> ws_order_number, RF008[bloom] -> tpcds_parquet.web_sales.ws_web_site_sk
| | stored statistics:
| | table: rows=719.38K size=45.09MB
| | columns: all
@@ -248,7 +248,7 @@ PLAN-ROOT SINK
| |
| 18:SCAN HDFS [tpcds_parquet.web_sales]
| HDFS partitions=1/1 files=2 size=45.09MB
-| runtime filters: RF008[bloom] -> tpcds_parquet.web_sales.ws_web_site_sk, RF010[bloom] -> tpcds_parquet.web_sales.ws_sold_date_sk
+| runtime filters: RF009[min_max] -> tpcds_parquet.web_sales.ws_web_site_sk, RF011[min_max] -> tpcds_parquet.web_sales.ws_sold_date_sk, RF008[bloom] -> tpcds_parquet.web_sales.ws_web_site_sk, RF010[bloom] -> tpcds_parquet.web_sales.ws_sold_date_sk
| stored statistics:
| table: rows=719.38K size=45.09MB
| columns: all
@@ -267,7 +267,7 @@ PLAN-ROOT SINK
| 15:HASH JOIN [INNER JOIN]
| | hash predicates: page_sk = cp_catalog_page_sk
| | fk/pk conjuncts: assumed fk/pk
-| | runtime filters: RF004[bloom] <- cp_catalog_page_sk
+| | runtime filters: RF004[bloom] <- cp_catalog_page_sk, RF005[min_max] <- cp_catalog_page_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=10,12,13 row-size=82B cardinality=1.59M
| | in pipelines: 10(GETNEXT), 11(GETNEXT), 13(OPEN)
@@ -285,7 +285,7 @@ PLAN-ROOT SINK
| 14:HASH JOIN [INNER JOIN]
| | hash predicates: date_sk = d_date_sk
| | fk/pk conjuncts: assumed fk/pk
-| | runtime filters: RF006[bloom] <- d_date_sk
+| | runtime filters: RF006[bloom] <- d_date_sk, RF007[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=10,12 row-size=50B cardinality=1.59M
| | in pipelines: 10(GETNEXT), 11(GETNEXT), 12(OPEN)
@@ -310,7 +310,7 @@ PLAN-ROOT SINK
| |
| |--11:SCAN HDFS [tpcds_parquet.catalog_returns]
| | HDFS partitions=1/1 files=1 size=10.62MB
-| | runtime filters: RF004[bloom] -> tpcds_parquet.catalog_returns.cr_catalog_page_sk, RF006[bloom] -> tpcds_parquet.catalog_returns.cr_returned_date_sk
+| | runtime filters: RF005[min_max] -> tpcds_parquet.catalog_returns.cr_catalog_page_sk, RF007[min_max] -> tpcds_parquet.catalog_returns.cr_returned_date_sk, RF004[bloom] -> tpcds_parquet.catalog_returns.cr_catalog_page_sk, RF006[bloom] -> tpcds_parquet.catalog_returns.cr_returned_date_sk
| | stored statistics:
| | table: rows=144.07K size=10.62MB
| | columns: all
@@ -321,7 +321,7 @@ PLAN-ROOT SINK
| |
| 10:SCAN HDFS [tpcds_parquet.catalog_sales]
| HDFS partitions=1/1 files=3 size=96.62MB
-| runtime filters: RF004[bloom] -> tpcds_parquet.catalog_sales.cs_catalog_page_sk, RF006[bloom] -> tpcds_parquet.catalog_sales.cs_sold_date_sk
+| runtime filters: RF005[min_max] -> tpcds_parquet.catalog_sales.cs_catalog_page_sk, RF007[min_max] -> tpcds_parquet.catalog_sales.cs_sold_date_sk, RF004[bloom] -> tpcds_parquet.catalog_sales.cs_catalog_page_sk, RF006[bloom] -> tpcds_parquet.catalog_sales.cs_sold_date_sk
| stored statistics:
| table: rows=1.44M size=96.62MB
| columns: all
@@ -340,7 +340,7 @@ PLAN-ROOT SINK
07:HASH JOIN [INNER JOIN]
| hash predicates: store_sk = s_store_sk
| fk/pk conjuncts: assumed fk/pk
-| runtime filters: RF000[bloom] <- s_store_sk
+| runtime filters: RF000[bloom] <- s_store_sk, RF001[min_max] <- s_store_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,4,5 row-size=82B cardinality=3.17M
| in pipelines: 02(GETNEXT), 03(GETNEXT), 05(OPEN)
@@ -358,7 +358,7 @@ PLAN-ROOT SINK
06:HASH JOIN [INNER JOIN]
| hash predicates: date_sk = d_date_sk
| fk/pk conjuncts: assumed fk/pk
-| runtime filters: RF002[bloom] <- d_date_sk
+| runtime filters: RF002[bloom] <- d_date_sk, RF003[min_max] <- d_date_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,4 row-size=50B cardinality=3.17M
| in pipelines: 02(GETNEXT), 03(GETNEXT), 04(OPEN)
@@ -383,7 +383,7 @@ PLAN-ROOT SINK
|
|--03:SCAN HDFS [tpcds_parquet.store_returns]
| HDFS partitions=1/1 files=1 size=15.43MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.store_returns.sr_store_sk, RF002[bloom] -> tpcds_parquet.store_returns.sr_returned_date_sk
+| runtime filters: RF001[min_max] -> tpcds_parquet.store_returns.sr_store_sk, RF003[min_max] -> tpcds_parquet.store_returns.sr_returned_date_sk, RF000[bloom] -> tpcds_parquet.store_returns.sr_store_sk, RF002[bloom] -> tpcds_parquet.store_returns.sr_returned_date_sk
| stored statistics:
| table: rows=287.51K size=15.43MB
| columns: all
@@ -393,10 +393,10 @@ PLAN-ROOT SINK
| in pipelines: 03(GETNEXT)
|
02:SCAN HDFS [tpcds_parquet.store_sales]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF000[bloom] -> tpcds_parquet.store_sales.ss_store_sk, RF002[bloom] -> tpcds_parquet.store_sales.ss_sold_date_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF001[min_max] -> tpcds_parquet.store_sales.ss_store_sk, RF000[bloom] -> tpcds_parquet.store_sales.ss_store_sk, RF002[bloom] -> tpcds_parquet.store_sales.ss_sold_date_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -498,7 +498,7 @@ Per-Host Resources: mem-estimate=40.62MB mem-reservation=8.94MB thread-reservati
| 25:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: wsr_web_site_sk = web_site_sk
| | fk/pk conjuncts: assumed fk/pk
-| | runtime filters: RF008[bloom] <- web_site_sk
+| | runtime filters: RF008[bloom] <- web_site_sk, RF009[min_max] <- web_site_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=19,21,22 row-size=82B cardinality=791.15K
| | in pipelines: 18(GETNEXT), 20(GETNEXT), 23(OPEN)
@@ -523,7 +523,7 @@ Per-Host Resources: mem-estimate=40.62MB mem-reservation=8.94MB thread-reservati
| 24:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: date_sk = d_date_sk
| | fk/pk conjuncts: assumed fk/pk
-| | runtime filters: RF010[bloom] <- d_date_sk
+| | runtime filters: RF010[bloom] <- d_date_sk, RF011[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=19,21 row-size=50B cardinality=791.15K
| | in pipelines: 18(GETNEXT), 20(GETNEXT), 22(OPEN)
@@ -556,7 +556,7 @@ Per-Host Resources: mem-estimate=40.62MB mem-reservation=8.94MB thread-reservati
| |--21:HASH JOIN [RIGHT OUTER JOIN, PARTITIONED]
| | | hash predicates: ws_item_sk = wr_item_sk, ws_order_number = wr_order_number
| | | fk/pk conjuncts: ws_item_sk = wr_item_sk, ws_order_number = wr_order_number
-| | | runtime filters: RF012[bloom] <- wr_item_sk, RF013[bloom] <- wr_order_number
+| | | runtime filters: RF012[bloom] <- wr_item_sk, RF013[bloom] <- wr_order_number, RF014[min_max] <- wr_item_sk, RF015[min_max] <- wr_order_number
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=18N,17 row-size=48B cardinality=71.76K
| | | in pipelines: 20(GETNEXT), 19(OPEN)
@@ -570,7 +570,7 @@ Per-Host Resources: mem-estimate=40.62MB mem-reservation=8.94MB thread-reservati
| | | Per-Host Resources: mem-estimate=81.00MB mem-reservation=3.00MB thread-reservation=2 runtime-filters-memory=1.00MB
| | | 19:SCAN HDFS [tpcds_parquet.web_returns, RANDOM]
| | | HDFS partitions=1/1 files=1 size=5.66MB
-| | | runtime filters: RF010[bloom] -> tpcds_parquet.web_returns.wr_returned_date_sk
+| | | runtime filters: RF011[min_max] -> tpcds_parquet.web_returns.wr_returned_date_sk, RF010[bloom] -> tpcds_parquet.web_returns.wr_returned_date_sk
| | | stored statistics:
| | | table: rows=71.76K size=5.66MB
| | | columns: all
@@ -588,7 +588,7 @@ Per-Host Resources: mem-estimate=40.62MB mem-reservation=8.94MB thread-reservati
| | Per-Host Resources: mem-estimate=99.00MB mem-reservation=7.00MB thread-reservation=2 runtime-filters-memory=3.00MB
| | 20:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| | HDFS partitions=1/1 files=2 size=45.09MB
-| | runtime filters: RF008[bloom] -> tpcds_parquet.web_sales.ws_web_site_sk, RF012[bloom] -> ws_item_sk, RF013[bloom] -> ws_order_number
+| | runtime filters: RF014[min_max] -> ws_item_sk, RF015[min_max] -> ws_order_number, RF009[min_max] -> tpcds_parquet.web_sales.ws_web_site_sk, RF012[bloom] -> ws_item_sk, RF013[bloom] -> ws_order_number, RF008[bloom] -> tpcds_parquet.web_sales.ws_web_site_sk
| | stored statistics:
| | table: rows=719.38K size=45.09MB
| | columns: all
@@ -599,7 +599,7 @@ Per-Host Resources: mem-estimate=40.62MB mem-reservation=8.94MB thread-reservati
| |
| 18:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| HDFS partitions=1/1 files=2 size=45.09MB
-| runtime filters: RF008[bloom] -> tpcds_parquet.web_sales.ws_web_site_sk, RF010[bloom] -> tpcds_parquet.web_sales.ws_sold_date_sk
+| runtime filters: RF009[min_max] -> tpcds_parquet.web_sales.ws_web_site_sk, RF011[min_max] -> tpcds_parquet.web_sales.ws_sold_date_sk, RF008[bloom] -> tpcds_parquet.web_sales.ws_web_site_sk, RF010[bloom] -> tpcds_parquet.web_sales.ws_sold_date_sk
| stored statistics:
| table: rows=719.38K size=45.09MB
| columns: all
@@ -632,7 +632,7 @@ Per-Host Resources: mem-estimate=40.62MB mem-reservation=8.94MB thread-reservati
| 15:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: page_sk = cp_catalog_page_sk
| | fk/pk conjuncts: assumed fk/pk
-| | runtime filters: RF004[bloom] <- cp_catalog_page_sk
+| | runtime filters: RF004[bloom] <- cp_catalog_page_sk, RF005[min_max] <- cp_catalog_page_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=10,12,13 row-size=82B cardinality=1.59M
| | in pipelines: 10(GETNEXT), 11(GETNEXT), 13(OPEN)
@@ -657,7 +657,7 @@ Per-Host Resources: mem-estimate=40.62MB mem-reservation=8.94MB thread-reservati
| 14:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: date_sk = d_date_sk
| | fk/pk conjuncts: assumed fk/pk
-| | runtime filters: RF006[bloom] <- d_date_sk
+| | runtime filters: RF006[bloom] <- d_date_sk, RF007[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=10,12 row-size=50B cardinality=1.59M
| | in pipelines: 10(GETNEXT), 11(GETNEXT), 12(OPEN)
@@ -689,7 +689,7 @@ Per-Host Resources: mem-estimate=40.62MB mem-reservation=8.94MB thread-reservati
| |
| |--11:SCAN HDFS [tpcds_parquet.catalog_returns, RANDOM]
| | HDFS partitions=1/1 files=1 size=10.62MB
-| | runtime filters: RF004[bloom] -> tpcds_parquet.catalog_returns.cr_catalog_page_sk, RF006[bloom] -> tpcds_parquet.catalog_returns.cr_returned_date_sk
+| | runtime filters: RF005[min_max] -> tpcds_parquet.catalog_returns.cr_catalog_page_sk, RF007[min_max] -> tpcds_parquet.catalog_returns.cr_returned_date_sk, RF004[bloom] -> tpcds_parquet.catalog_returns.cr_catalog_page_sk, RF006[bloom] -> tpcds_parquet.catalog_returns.cr_returned_date_sk
| | stored statistics:
| | table: rows=144.07K size=10.62MB
| | columns: all
@@ -700,7 +700,7 @@ Per-Host Resources: mem-estimate=40.62MB mem-reservation=8.94MB thread-reservati
| |
| 10:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| HDFS partitions=1/1 files=3 size=96.62MB
-| runtime filters: RF004[bloom] -> tpcds_parquet.catalog_sales.cs_catalog_page_sk, RF006[bloom] -> tpcds_parquet.catalog_sales.cs_sold_date_sk
+| runtime filters: RF005[min_max] -> tpcds_parquet.catalog_sales.cs_catalog_page_sk, RF007[min_max] -> tpcds_parquet.catalog_sales.cs_sold_date_sk, RF004[bloom] -> tpcds_parquet.catalog_sales.cs_catalog_page_sk, RF006[bloom] -> tpcds_parquet.catalog_sales.cs_sold_date_sk
| stored statistics:
| table: rows=1.44M size=96.62MB
| columns: all
@@ -733,7 +733,7 @@ Per-Host Resources: mem-estimate=112.10MB mem-reservation=11.88MB thread-reserva
07:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: store_sk = s_store_sk
| fk/pk conjuncts: assumed fk/pk
-| runtime filters: RF000[bloom] <- s_store_sk
+| runtime filters: RF000[bloom] <- s_store_sk, RF001[min_max] <- s_store_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,4,5 row-size=82B cardinality=3.17M
| in pipelines: 02(GETNEXT), 03(GETNEXT), 05(OPEN)
@@ -758,7 +758,7 @@ Per-Host Resources: mem-estimate=112.10MB mem-reservation=11.88MB thread-reserva
06:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: date_sk = d_date_sk
| fk/pk conjuncts: assumed fk/pk
-| runtime filters: RF002[bloom] <- d_date_sk
+| runtime filters: RF002[bloom] <- d_date_sk, RF003[min_max] <- d_date_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,4 row-size=50B cardinality=3.17M
| in pipelines: 02(GETNEXT), 03(GETNEXT), 04(OPEN)
@@ -790,7 +790,7 @@ Per-Host Resources: mem-estimate=112.10MB mem-reservation=11.88MB thread-reserva
|
|--03:SCAN HDFS [tpcds_parquet.store_returns, RANDOM]
| HDFS partitions=1/1 files=1 size=15.43MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.store_returns.sr_store_sk, RF002[bloom] -> tpcds_parquet.store_returns.sr_returned_date_sk
+| runtime filters: RF001[min_max] -> tpcds_parquet.store_returns.sr_store_sk, RF003[min_max] -> tpcds_parquet.store_returns.sr_returned_date_sk, RF000[bloom] -> tpcds_parquet.store_returns.sr_store_sk, RF002[bloom] -> tpcds_parquet.store_returns.sr_returned_date_sk
| stored statistics:
| table: rows=287.51K size=15.43MB
| columns: all
@@ -800,14 +800,482 @@ Per-Host Resources: mem-estimate=112.10MB mem-reservation=11.88MB thread-reserva
| in pipelines: 03(GETNEXT)
|
02:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF000[bloom] -> tpcds_parquet.store_sales.ss_store_sk, RF002[bloom] -> tpcds_parquet.store_sales.ss_sold_date_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF001[min_max] -> tpcds_parquet.store_sales.ss_store_sk, RF000[bloom] -> tpcds_parquet.store_sales.ss_store_sk, RF002[bloom] -> tpcds_parquet.store_sales.ss_sold_date_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
mem-estimate=48.00MB mem-reservation=2.00MB thread-reservation=1
tuple-ids=0 row-size=16B cardinality=2.88M
in pipelines: 02(GETNEXT)
+---- PARALLELPLANS
+Max Per-Host Resource Reservation: Memory=116.84MB Threads=24
+Per-Host Resource Estimates: Memory=519MB
+F22:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
+| Per-Instance Resources: mem-estimate=46.88KB mem-reservation=0B thread-reservation=1
+PLAN-ROOT SINK
+| output exprs: CASE valid_tid(27,28,29) WHEN 27 THEN channel WHEN 28 THEN channel WHEN 29 THEN NULL END, CASE valid_tid(27,28,29) WHEN 27 THEN id WHEN 28 THEN NULL WHEN 29 THEN NULL END, aggif(valid_tid(27,28,29) IN (27, 28, 29), CASE valid_tid(27,28,29) WHEN 27 THEN sum(sales) WHEN 28 THEN sum(sales) WHEN 29 THEN sum(sales) END), aggif(valid_tid(27,28,29) IN (27, 28, 29), CASE valid_tid(27,28,29) WHEN 27 THEN sum(`returns`) WHEN 28 THEN sum(`returns`) WHEN 29 THEN sum(`returns`) END), [...]
+| mem-estimate=0B mem-reservation=0B thread-reservation=0
+|
+46:MERGING-EXCHANGE [UNPARTITIONED]
+| order by: CASE valid_tid(27,28,29) WHEN 27 THEN channel WHEN 28 THEN channel WHEN 29 THEN NULL END ASC, CASE valid_tid(27,28,29) WHEN 27 THEN id WHEN 28 THEN NULL WHEN 29 THEN NULL END ASC
+| limit: 100
+| mem-estimate=46.88KB mem-reservation=0B thread-reservation=0
+| tuple-ids=31 row-size=72B cardinality=100
+| in pipelines: 29(GETNEXT)
+|
+F21:PLAN FRAGMENT [HASH(CASE valid_tid(27,28,29) WHEN 27 THEN murmur_hash(channel) WHEN 28 THEN murmur_hash(channel) WHEN 29 THEN murmur_hash(NULL) END,CASE valid_tid(27,28,29) WHEN 27 THEN murmur_hash(id) WHEN 28 THEN murmur_hash(NULL) WHEN 29 THEN murmur_hash(NULL) END)] hosts=3 instances=6
+Per-Instance Resources: mem-estimate=40.00MB mem-reservation=7.75MB thread-reservation=1
+29:TOP-N [LIMIT=100]
+| order by: CASE valid_tid(27,28,29) WHEN 27 THEN channel WHEN 28 THEN channel WHEN 29 THEN NULL END ASC, CASE valid_tid(27,28,29) WHEN 27 THEN id WHEN 28 THEN NULL WHEN 29 THEN NULL END ASC
+| mem-estimate=7.03KB mem-reservation=0B thread-reservation=0
+| tuple-ids=31 row-size=72B cardinality=100
+| in pipelines: 29(GETNEXT), 28(OPEN)
+|
+28:AGGREGATE [FINALIZE]
+| output: aggif(valid_tid(27,28,29) IN (CAST(27 AS INT), CAST(28 AS INT), CAST(29 AS INT)), CASE valid_tid(27,28,29) WHEN CAST(27 AS INT) THEN sum(sales) WHEN CAST(28 AS INT) THEN sum(sales) WHEN CAST(29 AS INT) THEN sum(sales) END), aggif(valid_tid(27,28,29) IN (CAST(27 AS INT), CAST(28 AS INT), CAST(29 AS INT)), CASE valid_tid(27,28,29) WHEN CAST(27 AS INT) THEN sum(`returns`) WHEN CAST(28 AS INT) THEN sum(`returns`) WHEN CAST(29 AS INT) THEN sum(`returns`) END), aggif(valid_tid(27,28 [...]
+| group by: CASE valid_tid(27,28,29) WHEN CAST(27 AS INT) THEN channel WHEN CAST(28 AS INT) THEN channel WHEN CAST(29 AS INT) THEN NULL END, CASE valid_tid(27,28,29) WHEN CAST(27 AS INT) THEN id WHEN CAST(28 AS INT) THEN NULL WHEN CAST(29 AS INT) THEN NULL END, CASE valid_tid(27,28,29) WHEN CAST(27 AS INT) THEN CAST(27 AS INT) WHEN CAST(28 AS INT) THEN CAST(28 AS INT) WHEN CAST(29 AS INT) THEN CAST(29 AS INT) END
+| mem-estimate=10.00MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=30 row-size=76B cardinality=11.56K
+| in pipelines: 28(GETNEXT), 45(OPEN)
+|
+45:AGGREGATE [FINALIZE]
+| Class 0
+| output: sum:merge(sales), sum:merge(`returns`), sum:merge(profit)
+| group by: channel, id
+| Class 1
+| output: sum:merge(sales), sum:merge(`returns`), sum:merge(profit)
+| group by: channel, NULL
+| Class 2
+| output: sum:merge(sales), sum:merge(`returns`), sum:merge(profit)
+| group by: NULL, NULL
+| mem-estimate=30.00MB mem-reservation=5.81MB thread-reservation=0
+| tuple-ids=27N,28N,29N row-size=216B cardinality=11.56K
+| in pipelines: 45(GETNEXT), 33(OPEN), 37(OPEN), 43(OPEN)
+|
+44:EXCHANGE [HASH(CASE valid_tid(27,28,29) WHEN 27 THEN murmur_hash(channel) WHEN 28 THEN murmur_hash(channel) WHEN 29 THEN murmur_hash(NULL) END,CASE valid_tid(27,28,29) WHEN 27 THEN murmur_hash(id) WHEN 28 THEN murmur_hash(NULL) WHEN 29 THEN murmur_hash(NULL) END)]
+| mem-estimate=2.13MB mem-reservation=0B thread-reservation=0
+| tuple-ids=27N,28N,29N row-size=216B cardinality=11.56K
+| in pipelines: 33(GETNEXT), 37(GETNEXT), 43(GETNEXT)
+|
+F20:PLAN FRAGMENT [RANDOM] hosts=3 instances=6
+Per-Instance Resources: mem-estimate=40.62MB mem-reservation=7.94MB thread-reservation=1
+27:AGGREGATE [STREAMING]
+| Class 0
+| output: sum(sales), sum(returns), sum(profit)
+| group by: channel, id
+| Class 1
+| output: sum(sales), sum(returns), sum(profit)
+| group by: channel, NULL
+| Class 2
+| output: sum(sales), sum(returns), sum(profit)
+| group by: NULL, NULL
+| mem-estimate=30.00MB mem-reservation=6.00MB thread-reservation=0
+| tuple-ids=27N,28N,29N row-size=216B cardinality=11.56K
+| in pipelines: 33(GETNEXT), 37(GETNEXT), 43(GETNEXT)
+|
+00:UNION
+| mem-estimate=0B mem-reservation=0B thread-reservation=0
+| tuple-ids=25 row-size=72B cardinality=11.56K
+| in pipelines: 33(GETNEXT), 37(GETNEXT), 43(GETNEXT)
+|
+|--43:AGGREGATE [FINALIZE]
+| | output: sum:merge(sales_price), sum:merge(profit), sum:merge(return_amt), sum:merge(net_loss)
+| | group by: web_site_id
+| | mem-estimate=10.00MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=23 row-size=92B cardinality=15
+| | in pipelines: 43(GETNEXT), 18(OPEN), 20(OPEN)
+| |
+| 42:EXCHANGE [HASH(web_site_id)]
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=23 row-size=92B cardinality=15
+| | in pipelines: 18(GETNEXT), 20(GETNEXT)
+| |
+| F16:PLAN FRAGMENT [RANDOM] hosts=2 instances=2
+| Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
+| Per-Instance Resources: mem-estimate=42.00MB mem-reservation=10.00MB thread-reservation=1
+| 26:AGGREGATE [STREAMING]
+| | output: sum(sales_price), sum(profit), sum(return_amt), sum(net_loss)
+| | group by: web_site_id
+| | mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=23 row-size=92B cardinality=15
+| | in pipelines: 18(GETNEXT), 20(GETNEXT)
+| |
+| 25:HASH JOIN [INNER JOIN, BROADCAST]
+| | hash-table-id=04
+| | hash predicates: wsr_web_site_sk = web_site_sk
+| | fk/pk conjuncts: assumed fk/pk
+| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=19,21,22 row-size=82B cardinality=791.15K
+| | in pipelines: 18(GETNEXT), 20(GETNEXT), 23(OPEN)
+| |
+| |--F27:PLAN FRAGMENT [RANDOM] hosts=2 instances=2
+| | | Per-Instance Resources: mem-estimate=4.89MB mem-reservation=4.88MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | JOIN BUILD
+| | | join-table-id=04 plan-id=05 cohort-id=03
+| | | build expressions: web_site_sk
+| | | runtime filters: RF008[bloom] <- web_site_sk, RF009[min_max] <- web_site_sk
+| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
+| | |
+| | 41:EXCHANGE [BROADCAST]
+| | | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=22 row-size=32B cardinality=30
+| | | in pipelines: 23(GETNEXT)
+| | |
+| | F18:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | Per-Instance Resources: mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=1
+| | 23:SCAN HDFS [tpcds_parquet.web_site, RANDOM]
+| | HDFS partitions=1/1 files=1 size=11.91KB
+| | stored statistics:
+| | table: rows=30 size=11.91KB
+| | columns: all
+| | extrapolated-rows=disabled max-scan-range-rows=30
+| | mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=0
+| | tuple-ids=22 row-size=32B cardinality=30
+| | in pipelines: 23(GETNEXT)
+| |
+| 24:HASH JOIN [INNER JOIN, BROADCAST]
+| | hash-table-id=05
+| | hash predicates: date_sk = d_date_sk
+| | fk/pk conjuncts: assumed fk/pk
+| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=19,21 row-size=50B cardinality=791.15K
+| | in pipelines: 18(GETNEXT), 20(GETNEXT), 22(OPEN)
+| |
+| |--F28:PLAN FRAGMENT [RANDOM] hosts=2 instances=2
+| | | Per-Instance Resources: mem-estimate=5.09MB mem-reservation=4.88MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | JOIN BUILD
+| | | join-table-id=05 plan-id=06 cohort-id=03
+| | | build expressions: d_date_sk
+| | | runtime filters: RF010[bloom] <- d_date_sk, RF011[min_max] <- d_date_sk
+| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
+| | |
+| | 40:EXCHANGE [BROADCAST]
+| | | mem-estimate=215.48KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=21 row-size=26B cardinality=7.30K
+| | | in pipelines: 22(GETNEXT)
+| | |
+| | F17:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | Per-Instance Resources: mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=1
+| | 22:SCAN HDFS [tpcds_parquet.date_dim, RANDOM]
+| | HDFS partitions=1/1 files=1 size=2.15MB
+| | predicates: CAST(d_date AS DATE) <= DATE '1998-08-18', CAST(d_date AS DATE) >= DATE '1998-08-04'
+| | stored statistics:
+| | table: rows=73.05K size=2.15MB
+| | columns: all
+| | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | parquet statistics predicates: CAST(d_date AS DATE) <= DATE '1998-08-18', CAST(d_date AS DATE) >= DATE '1998-08-04'
+| | parquet dictionary predicates: CAST(d_date AS DATE) <= DATE '1998-08-18', CAST(d_date AS DATE) >= DATE '1998-08-04'
+| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | tuple-ids=21 row-size=26B cardinality=7.30K
+| | in pipelines: 22(GETNEXT)
+| |
+| 17:UNION
+| | mem-estimate=0B mem-reservation=0B thread-reservation=0
+| | tuple-ids=19 row-size=24B cardinality=791.15K
+| | in pipelines: 18(GETNEXT), 20(GETNEXT)
+| |
+| |--21:HASH JOIN [RIGHT OUTER JOIN, PARTITIONED]
+| | | hash-table-id=06
+| | | hash predicates: ws_item_sk = wr_item_sk, ws_order_number = wr_order_number
+| | | fk/pk conjuncts: ws_item_sk = wr_item_sk, ws_order_number = wr_order_number
+| | | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| | | tuple-ids=18N,17 row-size=48B cardinality=71.76K
+| | | in pipelines: 20(GETNEXT), 19(OPEN)
+| | |
+| | |--F29:PLAN FRAGMENT [RANDOM] hosts=2 instances=2
+| | | | Per-Instance Resources: mem-estimate=5.89MB mem-reservation=3.94MB thread-reservation=1 runtime-filters-memory=2.00MB
+| | | JOIN BUILD
+| | | | join-table-id=06 plan-id=07 cohort-id=03
+| | | | build expressions: wr_item_sk, wr_order_number
+| | | | runtime filters: RF012[bloom] <- wr_item_sk, RF013[bloom] <- wr_order_number, RF014[min_max] <- wr_item_sk, RF015[min_max] <- wr_order_number
+| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| | | |
+| | | 39:EXCHANGE [HASH(wr_item_sk,wr_order_number)]
+| | | | mem-estimate=1.95MB mem-reservation=0B thread-reservation=0
+| | | | tuple-ids=17 row-size=28B cardinality=71.76K
+| | | | in pipelines: 19(GETNEXT)
+| | | |
+| | | F14:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | | Per-Host Shared Resources: mem-estimate=1.00MB mem-reservation=1.00MB thread-reservation=0 runtime-filters-memory=1.00MB
+| | | Per-Instance Resources: mem-estimate=16.00MB mem-reservation=2.00MB thread-reservation=1
+| | | 19:SCAN HDFS [tpcds_parquet.web_returns, RANDOM]
+| | | HDFS partitions=1/1 files=1 size=5.66MB
+| | | runtime filters: RF011[min_max] -> tpcds_parquet.web_returns.wr_returned_date_sk, RF010[bloom] -> tpcds_parquet.web_returns.wr_returned_date_sk
+| | | stored statistics:
+| | | table: rows=71.76K size=5.66MB
+| | | columns: all
+| | | extrapolated-rows=disabled max-scan-range-rows=71.76K
+| | | mem-estimate=16.00MB mem-reservation=2.00MB thread-reservation=0
+| | | tuple-ids=17 row-size=28B cardinality=71.76K
+| | | in pipelines: 19(GETNEXT)
+| | |
+| | 38:EXCHANGE [HASH(ws_item_sk,ws_order_number)]
+| | | mem-estimate=6.91MB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=18 row-size=20B cardinality=719.38K
+| | | in pipelines: 20(GETNEXT)
+| | |
+| | F13:PLAN FRAGMENT [RANDOM] hosts=2 instances=2
+| | Per-Host Shared Resources: mem-estimate=3.00MB mem-reservation=3.00MB thread-reservation=0 runtime-filters-memory=3.00MB
+| | Per-Instance Resources: mem-estimate=32.00MB mem-reservation=4.00MB thread-reservation=1
+| | 20:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
+| | HDFS partitions=1/1 files=2 size=45.09MB
+| | runtime filters: RF014[min_max] -> ws_item_sk, RF015[min_max] -> ws_order_number, RF009[min_max] -> tpcds_parquet.web_sales.ws_web_site_sk, RF012[bloom] -> ws_item_sk, RF013[bloom] -> ws_order_number, RF008[bloom] -> tpcds_parquet.web_sales.ws_web_site_sk
+| | stored statistics:
+| | table: rows=719.38K size=45.09MB
+| | columns: all
+| | extrapolated-rows=disabled max-scan-range-rows=644.77K
+| | mem-estimate=32.00MB mem-reservation=4.00MB thread-reservation=0
+| | tuple-ids=18 row-size=20B cardinality=719.38K
+| | in pipelines: 20(GETNEXT)
+| |
+| 18:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
+| HDFS partitions=1/1 files=2 size=45.09MB
+| runtime filters: RF009[min_max] -> tpcds_parquet.web_sales.ws_web_site_sk, RF011[min_max] -> tpcds_parquet.web_sales.ws_sold_date_sk, RF008[bloom] -> tpcds_parquet.web_sales.ws_web_site_sk, RF010[bloom] -> tpcds_parquet.web_sales.ws_sold_date_sk
+| stored statistics:
+| table: rows=719.38K size=45.09MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=644.77K
+| mem-estimate=32.00MB mem-reservation=8.00MB thread-reservation=0
+| tuple-ids=16 row-size=16B cardinality=719.38K
+| in pipelines: 18(GETNEXT)
+|
+|--37:AGGREGATE [FINALIZE]
+| | output: sum:merge(sales_price), sum:merge(profit), sum:merge(return_amt), sum:merge(net_loss)
+| | group by: cp_catalog_page_id
+| | mem-estimate=10.00MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=14 row-size=92B cardinality=11.54K
+| | in pipelines: 37(GETNEXT), 10(OPEN), 11(OPEN)
+| |
+| 36:EXCHANGE [HASH(cp_catalog_page_id)]
+| | mem-estimate=633.57KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=14 row-size=92B cardinality=11.54K
+| | in pipelines: 10(GETNEXT), 11(GETNEXT)
+| |
+| F08:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
+| Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
+| Per-Instance Resources: mem-estimate=58.00MB mem-reservation=11.00MB thread-reservation=1
+| 16:AGGREGATE [STREAMING]
+| | output: sum(sales_price), sum(profit), sum(return_amt), sum(net_loss)
+| | group by: cp_catalog_page_id
+| | mem-estimate=10.00MB mem-reservation=3.00MB spill-buffer=128.00KB thread-reservation=0
+| | tuple-ids=14 row-size=92B cardinality=11.54K
+| | in pipelines: 10(GETNEXT), 11(GETNEXT)
+| |
+| 15:HASH JOIN [INNER JOIN, BROADCAST]
+| | hash-table-id=02
+| | hash predicates: page_sk = cp_catalog_page_sk
+| | fk/pk conjuncts: assumed fk/pk
+| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=10,12,13 row-size=82B cardinality=1.59M
+| | in pipelines: 10(GETNEXT), 11(GETNEXT), 13(OPEN)
+| |
+| |--F25:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
+| | | Per-Instance Resources: mem-estimate=5.27MB mem-reservation=4.88MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | JOIN BUILD
+| | | join-table-id=02 plan-id=03 cohort-id=02
+| | | build expressions: cp_catalog_page_sk
+| | | runtime filters: RF004[bloom] <- cp_catalog_page_sk, RF005[min_max] <- cp_catalog_page_sk
+| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
+| | |
+| | 35:EXCHANGE [BROADCAST]
+| | | mem-estimate=402.19KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=13 row-size=32B cardinality=11.72K
+| | | in pipelines: 13(GETNEXT)
+| | |
+| | F10:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | Per-Instance Resources: mem-estimate=16.00MB mem-reservation=256.00KB thread-reservation=1
+| | 13:SCAN HDFS [tpcds_parquet.catalog_page, RANDOM]
+| | HDFS partitions=1/1 files=1 size=739.17KB
+| | stored statistics:
+| | table: rows=11.72K size=739.17KB
+| | columns: all
+| | extrapolated-rows=disabled max-scan-range-rows=11.72K
+| | mem-estimate=16.00MB mem-reservation=256.00KB thread-reservation=0
+| | tuple-ids=13 row-size=32B cardinality=11.72K
+| | in pipelines: 13(GETNEXT)
+| |
+| 14:HASH JOIN [INNER JOIN, BROADCAST]
+| | hash-table-id=03
+| | hash predicates: date_sk = d_date_sk
+| | fk/pk conjuncts: assumed fk/pk
+| | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| | tuple-ids=10,12 row-size=50B cardinality=1.59M
+| | in pipelines: 10(GETNEXT), 11(GETNEXT), 12(OPEN)
+| |
+| |--F26:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
+| | | Per-Instance Resources: mem-estimate=5.09MB mem-reservation=4.88MB thread-reservation=1 runtime-filters-memory=1.00MB
+| | JOIN BUILD
+| | | join-table-id=03 plan-id=04 cohort-id=02
+| | | build expressions: d_date_sk
+| | | runtime filters: RF006[bloom] <- d_date_sk, RF007[min_max] <- d_date_sk
+| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
+| | |
+| | 34:EXCHANGE [BROADCAST]
+| | | mem-estimate=215.48KB mem-reservation=0B thread-reservation=0
+| | | tuple-ids=12 row-size=26B cardinality=7.30K
+| | | in pipelines: 12(GETNEXT)
+| | |
+| | F09:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| | Per-Instance Resources: mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=1
+| | 12:SCAN HDFS [tpcds_parquet.date_dim, RANDOM]
+| | HDFS partitions=1/1 files=1 size=2.15MB
+| | predicates: CAST(d_date AS DATE) <= DATE '1998-08-18', CAST(d_date AS DATE) >= DATE '1998-08-04'
+| | stored statistics:
+| | table: rows=73.05K size=2.15MB
+| | columns: all
+| | extrapolated-rows=disabled max-scan-range-rows=73.05K
+| | parquet statistics predicates: CAST(d_date AS DATE) <= DATE '1998-08-18', CAST(d_date AS DATE) >= DATE '1998-08-04'
+| | parquet dictionary predicates: CAST(d_date AS DATE) <= DATE '1998-08-18', CAST(d_date AS DATE) >= DATE '1998-08-04'
+| | mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| | tuple-ids=12 row-size=26B cardinality=7.30K
+| | in pipelines: 12(GETNEXT)
+| |
+| 09:UNION
+| | mem-estimate=0B mem-reservation=0B thread-reservation=0
+| | tuple-ids=10 row-size=24B cardinality=1.59M
+| | in pipelines: 10(GETNEXT), 11(GETNEXT)
+| |
+| |--11:SCAN HDFS [tpcds_parquet.catalog_returns, RANDOM]
+| | HDFS partitions=1/1 files=1 size=10.62MB
+| | runtime filters: RF005[min_max] -> tpcds_parquet.catalog_returns.cr_catalog_page_sk, RF007[min_max] -> tpcds_parquet.catalog_returns.cr_returned_date_sk, RF004[bloom] -> tpcds_parquet.catalog_returns.cr_catalog_page_sk, RF006[bloom] -> tpcds_parquet.catalog_returns.cr_returned_date_sk
+| | stored statistics:
+| | table: rows=144.07K size=10.62MB
+| | columns: all
+| | extrapolated-rows=disabled max-scan-range-rows=144.07K
+| | mem-estimate=24.00MB mem-reservation=2.00MB thread-reservation=0
+| | tuple-ids=9 row-size=16B cardinality=144.07K
+| | in pipelines: 11(GETNEXT)
+| |
+| 10:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
+| HDFS partitions=1/1 files=3 size=96.62MB
+| runtime filters: RF005[min_max] -> tpcds_parquet.catalog_sales.cs_catalog_page_sk, RF007[min_max] -> tpcds_parquet.catalog_sales.cs_sold_date_sk, RF004[bloom] -> tpcds_parquet.catalog_sales.cs_catalog_page_sk, RF006[bloom] -> tpcds_parquet.catalog_sales.cs_sold_date_sk
+| stored statistics:
+| table: rows=1.44M size=96.62MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=650.14K
+| mem-estimate=48.00MB mem-reservation=8.00MB thread-reservation=0
+| tuple-ids=8 row-size=16B cardinality=1.44M
+| in pipelines: 10(GETNEXT)
+|
+33:AGGREGATE [FINALIZE]
+| output: sum:merge(sales_price), sum:merge(profit), sum:merge(return_amt), sum:merge(net_loss)
+| group by: s_store_id
+| mem-estimate=10.00MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=6 row-size=92B cardinality=6
+| in pipelines: 33(GETNEXT), 02(OPEN), 03(OPEN)
+|
+32:EXCHANGE [HASH(s_store_id)]
+| mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| tuple-ids=6 row-size=92B cardinality=6
+| in pipelines: 02(GETNEXT), 03(GETNEXT)
+|
+F02:PLAN FRAGMENT [RANDOM] hosts=3 instances=6
+Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB
+Per-Instance Resources: mem-estimate=34.00MB mem-reservation=6.00MB thread-reservation=1
+08:AGGREGATE [STREAMING]
+| output: sum(sales_price), sum(profit), sum(return_amt), sum(net_loss)
+| group by: s_store_id
+| mem-estimate=10.00MB mem-reservation=2.00MB spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=6 row-size=92B cardinality=6
+| in pipelines: 02(GETNEXT), 03(GETNEXT)
+|
+07:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=00
+| hash predicates: store_sk = s_store_sk
+| fk/pk conjuncts: assumed fk/pk
+| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=2,4,5 row-size=82B cardinality=3.17M
+| in pipelines: 02(GETNEXT), 03(GETNEXT), 05(OPEN)
+|
+|--F23:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
+| | Per-Instance Resources: mem-estimate=4.89MB mem-reservation=4.88MB thread-reservation=1 runtime-filters-memory=1.00MB
+| JOIN BUILD
+| | join-table-id=00 plan-id=01 cohort-id=01
+| | build expressions: s_store_sk
+| | runtime filters: RF000[bloom] <- s_store_sk, RF001[min_max] <- s_store_sk
+| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
+| |
+| 31:EXCHANGE [BROADCAST]
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=5 row-size=32B cardinality=12
+| | in pipelines: 05(GETNEXT)
+| |
+| F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Instance Resources: mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=1
+| 05:SCAN HDFS [tpcds_parquet.store, RANDOM]
+| HDFS partitions=1/1 files=1 size=9.93KB
+| stored statistics:
+| table: rows=12 size=9.93KB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=12
+| mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=0
+| tuple-ids=5 row-size=32B cardinality=12
+| in pipelines: 05(GETNEXT)
+|
+06:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=01
+| hash predicates: date_sk = d_date_sk
+| fk/pk conjuncts: assumed fk/pk
+| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=2,4 row-size=50B cardinality=3.17M
+| in pipelines: 02(GETNEXT), 03(GETNEXT), 04(OPEN)
+|
+|--F24:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
+| | Per-Instance Resources: mem-estimate=5.09MB mem-reservation=4.88MB thread-reservation=1 runtime-filters-memory=1.00MB
+| JOIN BUILD
+| | join-table-id=01 plan-id=02 cohort-id=01
+| | build expressions: d_date_sk
+| | runtime filters: RF002[bloom] <- d_date_sk, RF003[min_max] <- d_date_sk
+| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
+| |
+| 30:EXCHANGE [BROADCAST]
+| | mem-estimate=215.48KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=4 row-size=26B cardinality=7.30K
+| | in pipelines: 04(GETNEXT)
+| |
+| F03:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Instance Resources: mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=1
+| 04:SCAN HDFS [tpcds_parquet.date_dim, RANDOM]
+| HDFS partitions=1/1 files=1 size=2.15MB
+| predicates: CAST(d_date AS DATE) <= DATE '1998-08-18', CAST(d_date AS DATE) >= DATE '1998-08-04'
+| stored statistics:
+| table: rows=73.05K size=2.15MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=73.05K
+| parquet statistics predicates: CAST(d_date AS DATE) <= DATE '1998-08-18', CAST(d_date AS DATE) >= DATE '1998-08-04'
+| parquet dictionary predicates: CAST(d_date AS DATE) <= DATE '1998-08-18', CAST(d_date AS DATE) >= DATE '1998-08-04'
+| mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=0
+| tuple-ids=4 row-size=26B cardinality=7.30K
+| in pipelines: 04(GETNEXT)
+|
+01:UNION
+| mem-estimate=0B mem-reservation=0B thread-reservation=0
+| tuple-ids=2 row-size=24B cardinality=3.17M
+| in pipelines: 02(GETNEXT), 03(GETNEXT)
+|
+|--03:SCAN HDFS [tpcds_parquet.store_returns, RANDOM]
+| HDFS partitions=1/1 files=1 size=15.43MB
+| runtime filters: RF001[min_max] -> tpcds_parquet.store_returns.sr_store_sk, RF003[min_max] -> tpcds_parquet.store_returns.sr_returned_date_sk, RF000[bloom] -> tpcds_parquet.store_returns.sr_store_sk, RF002[bloom] -> tpcds_parquet.store_returns.sr_returned_date_sk
+| stored statistics:
+| table: rows=287.51K size=15.43MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=287.51K
+| mem-estimate=24.00MB mem-reservation=4.00MB thread-reservation=0
+| tuple-ids=1 row-size=16B cardinality=287.51K
+| in pipelines: 03(GETNEXT)
+|
+02:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF001[min_max] -> tpcds_parquet.store_sales.ss_store_sk, RF000[bloom] -> tpcds_parquet.store_sales.ss_store_sk, RF002[bloom] -> tpcds_parquet.store_sales.ss_sold_date_sk
+ stored statistics:
+ table: rows=2.88M size=200.96MB
+ partitions: 1824/1824 rows=2.88M
+ columns: all
+ extrapolated-rows=disabled max-scan-range-rows=130.09K
+ mem-estimate=16.00MB mem-reservation=2.00MB thread-reservation=0
+ tuple-ids=0 row-size=16B cardinality=2.88M
+ in pipelines: 02(GETNEXT)
====
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q06.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q06.test
index 9b6c815..1c326a3 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q06.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q06.test
@@ -50,7 +50,7 @@ PLAN-ROOT SINK
14:HASH JOIN [LEFT SEMI JOIN]
| hash predicates: i.i_category = j.i_category
| other join predicates: i.i_current_price > CAST(1.2 AS DECIMAL(2,1)) * avg(j.i_current_price)
-| runtime filters: RF000[bloom] <- j.i_category
+| runtime filters: RF000[bloom] <- j.i_category, RF001[min_max] <- j.i_category
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,1,0,3,4 row-size=80B cardinality=1.19K
| in pipelines: 02(GETNEXT), 08(OPEN)
@@ -74,7 +74,7 @@ PLAN-ROOT SINK
|
13:HASH JOIN [LEFT SEMI JOIN]
| hash predicates: d.d_month_seq = (d_month_seq)
-| runtime filters: RF002[bloom] <- (d_month_seq)
+| runtime filters: RF002[bloom] <- (d_month_seq), RF003[min_max] <- (d_month_seq)
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,1,0,3,4 row-size=80B cardinality=1.19K
| in pipelines: 02(GETNEXT), 06(OPEN)
@@ -102,14 +102,14 @@ PLAN-ROOT SINK
12:HASH JOIN [INNER JOIN]
| hash predicates: s.ss_item_sk = i.i_item_sk
| fk/pk conjuncts: s.ss_item_sk = i.i_item_sk
-| runtime filters: RF004[bloom] <- i.i_item_sk
+| runtime filters: RF004[bloom] <- i.i_item_sk, RF005[min_max] <- i.i_item_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,1,0,3,4 row-size=80B cardinality=2.88M
| in pipelines: 02(GETNEXT), 04(OPEN)
|
|--04:SCAN HDFS [tpcds_parquet.item i]
| HDFS partitions=1/1 files=1 size=1.73MB
-| runtime filters: RF000[bloom] -> i.i_category
+| runtime filters: RF001[min_max] -> i.i_category, RF000[bloom] -> i.i_category
| stored statistics:
| table: rows=18.00K size=1.73MB
| columns: all
@@ -128,7 +128,7 @@ PLAN-ROOT SINK
|
|--03:SCAN HDFS [tpcds_parquet.date_dim d]
| HDFS partitions=1/1 files=1 size=2.15MB
-| runtime filters: RF002[bloom] -> d.d_month_seq
+| runtime filters: RF003[min_max] -> d.d_month_seq, RF002[bloom] -> d.d_month_seq
| stored statistics:
| table: rows=73.05K size=2.15MB
| columns: all
@@ -140,7 +140,7 @@ PLAN-ROOT SINK
10:HASH JOIN [INNER JOIN]
| hash predicates: c.c_current_addr_sk = a.ca_address_sk
| fk/pk conjuncts: c.c_current_addr_sk = a.ca_address_sk
-| runtime filters: RF008[bloom] <- a.ca_address_sk
+| runtime filters: RF008[bloom] <- a.ca_address_sk, RF009[min_max] <- a.ca_address_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,1,0 row-size=42B cardinality=2.88M
| in pipelines: 02(GETNEXT), 00(OPEN)
@@ -158,14 +158,14 @@ PLAN-ROOT SINK
09:HASH JOIN [INNER JOIN]
| hash predicates: s.ss_customer_sk = c.c_customer_sk
| fk/pk conjuncts: s.ss_customer_sk = c.c_customer_sk
-| runtime filters: RF010[bloom] <- c.c_customer_sk
+| runtime filters: RF010[bloom] <- c.c_customer_sk, RF011[min_max] <- c.c_customer_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,1 row-size=24B cardinality=2.88M
| in pipelines: 02(GETNEXT), 01(OPEN)
|
|--01:SCAN HDFS [tpcds_parquet.customer c]
| HDFS partitions=1/1 files=1 size=5.49MB
-| runtime filters: RF008[bloom] -> c.c_current_addr_sk
+| runtime filters: RF009[min_max] -> c.c_current_addr_sk, RF008[bloom] -> c.c_current_addr_sk
| stored statistics:
| table: rows=100.00K size=5.49MB
| columns: all
@@ -175,10 +175,10 @@ PLAN-ROOT SINK
| in pipelines: 01(GETNEXT)
|
02:SCAN HDFS [tpcds_parquet.store_sales s]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF004[bloom] -> s.ss_item_sk, RF006[bloom] -> s.ss_sold_date_sk, RF010[bloom] -> s.ss_customer_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF005[min_max] -> s.ss_item_sk, RF011[min_max] -> s.ss_customer_sk, RF004[bloom] -> s.ss_item_sk, RF006[bloom] -> s.ss_sold_date_sk, RF010[bloom] -> s.ss_customer_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -234,7 +234,7 @@ Per-Host Resources: mem-estimate=62.43MB mem-reservation=20.62MB thread-reservat
14:HASH JOIN [LEFT SEMI JOIN, BROADCAST]
| hash predicates: i.i_category = j.i_category
| other join predicates: i.i_current_price > CAST(1.2 AS DECIMAL(2,1)) * avg(j.i_current_price)
-| runtime filters: RF000[bloom] <- j.i_category
+| runtime filters: RF000[bloom] <- j.i_category, RF001[min_max] <- j.i_category
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,1,0,3,4 row-size=80B cardinality=1.19K
| in pipelines: 02(GETNEXT), 26(OPEN)
@@ -279,7 +279,7 @@ Per-Host Resources: mem-estimate=62.43MB mem-reservation=20.62MB thread-reservat
|
13:HASH JOIN [LEFT SEMI JOIN, BROADCAST]
| hash predicates: d.d_month_seq = (d_month_seq)
-| runtime filters: RF002[bloom] <- (d_month_seq)
+| runtime filters: RF002[bloom] <- (d_month_seq), RF003[min_max] <- (d_month_seq)
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,1,0,3,4 row-size=80B cardinality=1.19K
| in pipelines: 02(GETNEXT), 22(OPEN)
@@ -335,7 +335,7 @@ Per-Host Resources: mem-estimate=62.43MB mem-reservation=20.62MB thread-reservat
12:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: s.ss_item_sk = i.i_item_sk
| fk/pk conjuncts: s.ss_item_sk = i.i_item_sk
-| runtime filters: RF004[bloom] <- i.i_item_sk
+| runtime filters: RF004[bloom] <- i.i_item_sk, RF005[min_max] <- i.i_item_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,1,0,3,4 row-size=80B cardinality=2.88M
| in pipelines: 02(GETNEXT), 04(OPEN)
@@ -349,7 +349,7 @@ Per-Host Resources: mem-estimate=62.43MB mem-reservation=20.62MB thread-reservat
| Per-Host Resources: mem-estimate=49.00MB mem-reservation=1.25MB thread-reservation=2 runtime-filters-memory=1.00MB
| 04:SCAN HDFS [tpcds_parquet.item i, RANDOM]
| HDFS partitions=1/1 files=1 size=1.73MB
-| runtime filters: RF000[bloom] -> i.i_category
+| runtime filters: RF001[min_max] -> i.i_category, RF000[bloom] -> i.i_category
| stored statistics:
| table: rows=18.00K size=1.73MB
| columns: all
@@ -375,7 +375,7 @@ Per-Host Resources: mem-estimate=62.43MB mem-reservation=20.62MB thread-reservat
| Per-Host Resources: mem-estimate=33.00MB mem-reservation=1.50MB thread-reservation=2 runtime-filters-memory=1.00MB
| 03:SCAN HDFS [tpcds_parquet.date_dim d, RANDOM]
| HDFS partitions=1/1 files=1 size=2.15MB
-| runtime filters: RF002[bloom] -> d.d_month_seq
+| runtime filters: RF003[min_max] -> d.d_month_seq, RF002[bloom] -> d.d_month_seq
| stored statistics:
| table: rows=73.05K size=2.15MB
| columns: all
@@ -387,7 +387,7 @@ Per-Host Resources: mem-estimate=62.43MB mem-reservation=20.62MB thread-reservat
10:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: c.c_current_addr_sk = a.ca_address_sk
| fk/pk conjuncts: c.c_current_addr_sk = a.ca_address_sk
-| runtime filters: RF008[bloom] <- a.ca_address_sk
+| runtime filters: RF008[bloom] <- a.ca_address_sk, RF009[min_max] <- a.ca_address_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,1,0 row-size=42B cardinality=2.88M
| in pipelines: 02(GETNEXT), 00(OPEN)
@@ -412,7 +412,7 @@ Per-Host Resources: mem-estimate=62.43MB mem-reservation=20.62MB thread-reservat
09:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: s.ss_customer_sk = c.c_customer_sk
| fk/pk conjuncts: s.ss_customer_sk = c.c_customer_sk
-| runtime filters: RF010[bloom] <- c.c_customer_sk
+| runtime filters: RF010[bloom] <- c.c_customer_sk, RF011[min_max] <- c.c_customer_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,1 row-size=24B cardinality=2.88M
| in pipelines: 02(GETNEXT), 01(OPEN)
@@ -426,7 +426,7 @@ Per-Host Resources: mem-estimate=62.43MB mem-reservation=20.62MB thread-reservat
| Per-Host Resources: mem-estimate=33.00MB mem-reservation=2.00MB thread-reservation=2 runtime-filters-memory=1.00MB
| 01:SCAN HDFS [tpcds_parquet.customer c, RANDOM]
| HDFS partitions=1/1 files=1 size=5.49MB
-| runtime filters: RF008[bloom] -> c.c_current_addr_sk
+| runtime filters: RF009[min_max] -> c.c_current_addr_sk, RF008[bloom] -> c.c_current_addr_sk
| stored statistics:
| table: rows=100.00K size=5.49MB
| columns: all
@@ -436,10 +436,10 @@ Per-Host Resources: mem-estimate=62.43MB mem-reservation=20.62MB thread-reservat
| in pipelines: 01(GETNEXT)
|
02:SCAN HDFS [tpcds_parquet.store_sales s, RANDOM]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF004[bloom] -> s.ss_item_sk, RF006[bloom] -> s.ss_sold_date_sk, RF010[bloom] -> s.ss_customer_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF005[min_max] -> s.ss_item_sk, RF011[min_max] -> s.ss_customer_sk, RF004[bloom] -> s.ss_item_sk, RF006[bloom] -> s.ss_sold_date_sk, RF010[bloom] -> s.ss_customer_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -506,7 +506,7 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=3.00MB thread-reser
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: j.i_category
-| | runtime filters: RF000[bloom] <- j.i_category
+| | runtime filters: RF000[bloom] <- j.i_category, RF001[min_max] <- j.i_category
| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| |
| 27:EXCHANGE [BROADCAST]
@@ -559,7 +559,7 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=3.00MB thread-reser
| JOIN BUILD
| | join-table-id=01 plan-id=02 cohort-id=01
| | build expressions: (d_month_seq)
-| | runtime filters: RF002[bloom] <- (d_month_seq)
+| | runtime filters: RF002[bloom] <- (d_month_seq), RF003[min_max] <- (d_month_seq)
| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| |
| 24:EXCHANGE [BROADCAST]
@@ -623,7 +623,7 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=3.00MB thread-reser
| JOIN BUILD
| | join-table-id=02 plan-id=03 cohort-id=01
| | build expressions: i.i_item_sk
-| | runtime filters: RF004[bloom] <- i.i_item_sk
+| | runtime filters: RF004[bloom] <- i.i_item_sk, RF005[min_max] <- i.i_item_sk
| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| |
| 20:EXCHANGE [BROADCAST]
@@ -636,7 +636,7 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=3.00MB thread-reser
| Per-Instance Resources: mem-estimate=16.00MB mem-reservation=256.00KB thread-reservation=1
| 04:SCAN HDFS [tpcds_parquet.item i, RANDOM]
| HDFS partitions=1/1 files=1 size=1.73MB
-| runtime filters: RF000[bloom] -> i.i_category
+| runtime filters: RF001[min_max] -> i.i_category, RF000[bloom] -> i.i_category
| stored statistics:
| table: rows=18.00K size=1.73MB
| columns: all
@@ -671,7 +671,7 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=3.00MB thread-reser
| Per-Instance Resources: mem-estimate=16.00MB mem-reservation=512.00KB thread-reservation=1
| 03:SCAN HDFS [tpcds_parquet.date_dim d, RANDOM]
| HDFS partitions=1/1 files=1 size=2.15MB
-| runtime filters: RF002[bloom] -> d.d_month_seq
+| runtime filters: RF003[min_max] -> d.d_month_seq, RF002[bloom] -> d.d_month_seq
| stored statistics:
| table: rows=73.05K size=2.15MB
| columns: all
@@ -693,7 +693,7 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=3.00MB thread-reser
| JOIN BUILD
| | join-table-id=04 plan-id=05 cohort-id=01
| | build expressions: a.ca_address_sk
-| | runtime filters: RF008[bloom] <- a.ca_address_sk
+| | runtime filters: RF008[bloom] <- a.ca_address_sk, RF009[min_max] <- a.ca_address_sk
| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| |
| 18:EXCHANGE [BROADCAST]
@@ -726,7 +726,7 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=3.00MB thread-reser
| JOIN BUILD
| | join-table-id=05 plan-id=06 cohort-id=01
| | build expressions: c.c_customer_sk
-| | runtime filters: RF010[bloom] <- c.c_customer_sk
+| | runtime filters: RF010[bloom] <- c.c_customer_sk, RF011[min_max] <- c.c_customer_sk
| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| |
| 17:EXCHANGE [BROADCAST]
@@ -739,7 +739,7 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=3.00MB thread-reser
| Per-Instance Resources: mem-estimate=16.00MB mem-reservation=1.00MB thread-reservation=1
| 01:SCAN HDFS [tpcds_parquet.customer c, RANDOM]
| HDFS partitions=1/1 files=1 size=5.49MB
-| runtime filters: RF008[bloom] -> c.c_current_addr_sk
+| runtime filters: RF009[min_max] -> c.c_current_addr_sk, RF008[bloom] -> c.c_current_addr_sk
| stored statistics:
| table: rows=100.00K size=5.49MB
| columns: all
@@ -749,10 +749,10 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=3.00MB thread-reser
| in pipelines: 01(GETNEXT)
|
02:SCAN HDFS [tpcds_parquet.store_sales s, RANDOM]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF004[bloom] -> s.ss_item_sk, RF006[bloom] -> s.ss_sold_date_sk, RF010[bloom] -> s.ss_customer_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF005[min_max] -> s.ss_item_sk, RF011[min_max] -> s.ss_customer_sk, RF004[bloom] -> s.ss_item_sk, RF006[bloom] -> s.ss_sold_date_sk, RF010[bloom] -> s.ss_customer_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q07.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q07.test
index 91e1a41..af23a74 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q07.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q07.test
@@ -52,7 +52,7 @@ PLAN-ROOT SINK
08:HASH JOIN [INNER JOIN]
| hash predicates: ss_item_sk = i_item_sk
| fk/pk conjuncts: ss_item_sk = i_item_sk
-| runtime filters: RF000[bloom] <- i_item_sk
+| runtime filters: RF000[bloom] <- i_item_sk, RF001[min_max] <- i_item_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=0,2,1,4,3 row-size=162B cardinality=261.60K
| in pipelines: 00(GETNEXT), 03(OPEN)
@@ -70,7 +70,7 @@ PLAN-ROOT SINK
07:HASH JOIN [INNER JOIN]
| hash predicates: ss_promo_sk = p_promo_sk
| fk/pk conjuncts: ss_promo_sk = p_promo_sk
-| runtime filters: RF002[bloom] <- p_promo_sk
+| runtime filters: RF002[bloom] <- p_promo_sk, RF003[min_max] <- p_promo_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=0,2,1,4 row-size=126B cardinality=261.60K
| in pipelines: 00(GETNEXT), 04(OPEN)
@@ -89,7 +89,7 @@ PLAN-ROOT SINK
06:HASH JOIN [INNER JOIN]
| hash predicates: ss_cdemo_sk = cd_demo_sk
| fk/pk conjuncts: ss_cdemo_sk = cd_demo_sk
-| runtime filters: RF004[bloom] <- cd_demo_sk
+| runtime filters: RF004[bloom] <- cd_demo_sk, RF005[min_max] <- cd_demo_sk
| mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| tuple-ids=0,2,1 row-size=96B cardinality=263.34K
| in pipelines: 00(GETNEXT), 01(OPEN)
@@ -129,10 +129,10 @@ PLAN-ROOT SINK
| in pipelines: 02(GETNEXT)
|
00:SCAN HDFS [tpcds_parquet.store_sales]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF000[bloom] -> ss_item_sk, RF002[bloom] -> ss_promo_sk, RF004[bloom] -> ss_cdemo_sk, RF006[bloom] -> ss_sold_date_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF001[min_max] -> ss_item_sk, RF003[min_max] -> ss_promo_sk, RF005[min_max] -> ss_cdemo_sk, RF000[bloom] -> ss_item_sk, RF002[bloom] -> ss_promo_sk, RF004[bloom] -> ss_cdemo_sk, RF006[bloom] -> ss_sold_date_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -187,7 +187,7 @@ Per-Host Resources: mem-estimate=145.85MB mem-reservation=24.31MB thread-reserva
08:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: ss_item_sk = i_item_sk
| fk/pk conjuncts: ss_item_sk = i_item_sk
-| runtime filters: RF000[bloom] <- i_item_sk
+| runtime filters: RF000[bloom] <- i_item_sk, RF001[min_max] <- i_item_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=0,2,1,4,3 row-size=162B cardinality=261.60K
| in pipelines: 00(GETNEXT), 03(OPEN)
@@ -212,7 +212,7 @@ Per-Host Resources: mem-estimate=145.85MB mem-reservation=24.31MB thread-reserva
07:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: ss_promo_sk = p_promo_sk
| fk/pk conjuncts: ss_promo_sk = p_promo_sk
-| runtime filters: RF002[bloom] <- p_promo_sk
+| runtime filters: RF002[bloom] <- p_promo_sk, RF003[min_max] <- p_promo_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=0,2,1,4 row-size=126B cardinality=261.60K
| in pipelines: 00(GETNEXT), 04(OPEN)
@@ -238,7 +238,7 @@ Per-Host Resources: mem-estimate=145.85MB mem-reservation=24.31MB thread-reserva
06:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: ss_cdemo_sk = cd_demo_sk
| fk/pk conjuncts: ss_cdemo_sk = cd_demo_sk
-| runtime filters: RF004[bloom] <- cd_demo_sk
+| runtime filters: RF004[bloom] <- cd_demo_sk, RF005[min_max] <- cd_demo_sk
| mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| tuple-ids=0,2,1 row-size=96B cardinality=263.34K
| in pipelines: 00(GETNEXT), 01(OPEN)
@@ -292,10 +292,10 @@ Per-Host Resources: mem-estimate=145.85MB mem-reservation=24.31MB thread-reserva
| in pipelines: 02(GETNEXT)
|
00:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF000[bloom] -> ss_item_sk, RF002[bloom] -> ss_promo_sk, RF004[bloom] -> ss_cdemo_sk, RF006[bloom] -> ss_sold_date_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF001[min_max] -> ss_item_sk, RF003[min_max] -> ss_promo_sk, RF005[min_max] -> ss_cdemo_sk, RF000[bloom] -> ss_item_sk, RF002[bloom] -> ss_promo_sk, RF004[bloom] -> ss_cdemo_sk, RF006[bloom] -> ss_sold_date_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -361,7 +361,7 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=6.00MB thread-reser
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: i_item_sk
-| | runtime filters: RF000[bloom] <- i_item_sk
+| | runtime filters: RF000[bloom] <- i_item_sk, RF001[min_max] <- i_item_sk
| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| |
| 14:EXCHANGE [BROADCAST]
@@ -394,7 +394,7 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=6.00MB thread-reser
| JOIN BUILD
| | join-table-id=01 plan-id=02 cohort-id=01
| | build expressions: p_promo_sk
-| | runtime filters: RF002[bloom] <- p_promo_sk
+| | runtime filters: RF002[bloom] <- p_promo_sk, RF003[min_max] <- p_promo_sk
| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| |
| 13:EXCHANGE [BROADCAST]
@@ -428,7 +428,7 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=6.00MB thread-reser
| JOIN BUILD
| | join-table-id=02 plan-id=03 cohort-id=01
| | build expressions: cd_demo_sk
-| | runtime filters: RF004[bloom] <- cd_demo_sk
+| | runtime filters: RF004[bloom] <- cd_demo_sk, RF005[min_max] <- cd_demo_sk
| | mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=512.00KB thread-reservation=0
| |
| 12:EXCHANGE [BROADCAST]
@@ -488,10 +488,10 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=6.00MB thread-reser
| in pipelines: 02(GETNEXT)
|
00:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF000[bloom] -> ss_item_sk, RF002[bloom] -> ss_promo_sk, RF004[bloom] -> ss_cdemo_sk, RF006[bloom] -> ss_sold_date_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF001[min_max] -> ss_item_sk, RF003[min_max] -> ss_promo_sk, RF005[min_max] -> ss_cdemo_sk, RF000[bloom] -> ss_item_sk, RF002[bloom] -> ss_promo_sk, RF004[bloom] -> ss_cdemo_sk, RF006[bloom] -> ss_sold_date_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q08.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q08.test
index fe50c6a..d5f918e 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q08.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q08.test
@@ -124,7 +124,7 @@ PLAN-ROOT SINK
| | 07:HASH JOIN [INNER JOIN]
| | | hash predicates: ca_address_sk = c_current_addr_sk
| | | fk/pk conjuncts: none
-| | | runtime filters: RF006[bloom] <- c_current_addr_sk
+| | | runtime filters: RF006[bloom] <- c_current_addr_sk, RF007[min_max] <- c_current_addr_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=6,7 row-size=38B cardinality=51.30K
| | | in pipelines: 05(GETNEXT), 06(OPEN)
@@ -144,7 +144,7 @@ PLAN-ROOT SINK
| | |
| | 05:SCAN HDFS [tpcds_parquet.customer_address]
| | HDFS partitions=1/1 files=1 size=1.16MB
-| | runtime filters: RF006[bloom] -> ca_address_sk
+| | runtime filters: RF007[min_max] -> ca_address_sk, RF006[bloom] -> ca_address_sk
| | stored statistics:
| | table: rows=50.00K size=1.16MB
| | columns: all
@@ -174,7 +174,7 @@ PLAN-ROOT SINK
11:HASH JOIN [INNER JOIN]
| hash predicates: ss_store_sk = s_store_sk
| fk/pk conjuncts: ss_store_sk = s_store_sk
-| runtime filters: RF002[bloom] <- s_store_sk
+| runtime filters: RF002[bloom] <- s_store_sk, RF003[min_max] <- s_store_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=0,1,2 row-size=61B cardinality=293.73K
| in pipelines: 00(GETNEXT), 02(OPEN)
@@ -212,10 +212,10 @@ PLAN-ROOT SINK
| in pipelines: 01(GETNEXT)
|
00:SCAN HDFS [tpcds_parquet.store_sales]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF002[bloom] -> ss_store_sk, RF004[bloom] -> ss_sold_date_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF003[min_max] -> ss_store_sk, RF002[bloom] -> ss_store_sk, RF004[bloom] -> ss_sold_date_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -313,7 +313,7 @@ Per-Host Resources: mem-estimate=50.86MB mem-reservation=11.81MB thread-reservat
| | 07:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash predicates: ca_address_sk = c_current_addr_sk
| | | fk/pk conjuncts: none
-| | | runtime filters: RF006[bloom] <- c_current_addr_sk
+| | | runtime filters: RF006[bloom] <- c_current_addr_sk, RF007[min_max] <- c_current_addr_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=6,7 row-size=38B cardinality=51.30K
| | | in pipelines: 05(GETNEXT), 06(OPEN)
@@ -340,7 +340,7 @@ Per-Host Resources: mem-estimate=50.86MB mem-reservation=11.81MB thread-reservat
| | |
| | 05:SCAN HDFS [tpcds_parquet.customer_address, RANDOM]
| | HDFS partitions=1/1 files=1 size=1.16MB
-| | runtime filters: RF006[bloom] -> ca_address_sk
+| | runtime filters: RF007[min_max] -> ca_address_sk, RF006[bloom] -> ca_address_sk
| | stored statistics:
| | table: rows=50.00K size=1.16MB
| | columns: all
@@ -383,7 +383,7 @@ Per-Host Resources: mem-estimate=50.86MB mem-reservation=11.81MB thread-reservat
11:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: ss_store_sk = s_store_sk
| fk/pk conjuncts: ss_store_sk = s_store_sk
-| runtime filters: RF002[bloom] <- s_store_sk
+| runtime filters: RF002[bloom] <- s_store_sk, RF003[min_max] <- s_store_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=0,1,2 row-size=61B cardinality=293.73K
| in pipelines: 00(GETNEXT), 02(OPEN)
@@ -435,10 +435,10 @@ Per-Host Resources: mem-estimate=50.86MB mem-reservation=11.81MB thread-reservat
| in pipelines: 01(GETNEXT)
|
00:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF002[bloom] -> ss_store_sk, RF004[bloom] -> ss_sold_date_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF003[min_max] -> ss_store_sk, RF002[bloom] -> ss_store_sk, RF004[bloom] -> ss_sold_date_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -564,7 +564,7 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=3.00MB thread-reser
| | | JOIN BUILD
| | | | join-table-id=02 plan-id=03 cohort-id=03
| | | | build expressions: c_current_addr_sk
-| | | | runtime filters: RF006[bloom] <- c_current_addr_sk
+| | | | runtime filters: RF006[bloom] <- c_current_addr_sk, RF007[min_max] <- c_current_addr_sk
| | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | |
| | | 19:EXCHANGE [BROADCAST]
@@ -589,7 +589,7 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=3.00MB thread-reser
| | |
| | 05:SCAN HDFS [tpcds_parquet.customer_address, RANDOM]
| | HDFS partitions=1/1 files=1 size=1.16MB
-| | runtime filters: RF006[bloom] -> ca_address_sk
+| | runtime filters: RF007[min_max] -> ca_address_sk, RF006[bloom] -> ca_address_sk
| | stored statistics:
| | table: rows=50.00K size=1.16MB
| | columns: all
@@ -642,7 +642,7 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=3.00MB thread-reser
| JOIN BUILD
| | join-table-id=03 plan-id=04 cohort-id=01
| | build expressions: s_store_sk
-| | runtime filters: RF002[bloom] <- s_store_sk
+| | runtime filters: RF002[bloom] <- s_store_sk, RF003[min_max] <- s_store_sk
| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| |
| 16:EXCHANGE [BROADCAST]
@@ -701,10 +701,10 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=3.00MB thread-reser
| in pipelines: 01(GETNEXT)
|
00:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF002[bloom] -> ss_store_sk, RF004[bloom] -> ss_sold_date_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF003[min_max] -> ss_store_sk, RF002[bloom] -> ss_store_sk, RF004[bloom] -> ss_sold_date_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q09.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q09.test
index 73f31e7..30966a1 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q09.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q09.test
@@ -67,10 +67,10 @@ PLAN-ROOT SINK
| | in pipelines: 30(GETNEXT), 29(OPEN)
| |
| 29:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(100 AS INT), ss_quantity >= CAST(81 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -92,10 +92,10 @@ PLAN-ROOT SINK
| | in pipelines: 28(GETNEXT), 27(OPEN)
| |
| 27:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(100 AS INT), ss_quantity >= CAST(81 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -117,10 +117,10 @@ PLAN-ROOT SINK
| | in pipelines: 26(GETNEXT), 25(OPEN)
| |
| 25:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(100 AS INT), ss_quantity >= CAST(81 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -142,10 +142,10 @@ PLAN-ROOT SINK
| | in pipelines: 24(GETNEXT), 23(OPEN)
| |
| 23:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(80 AS INT), ss_quantity >= CAST(61 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -167,10 +167,10 @@ PLAN-ROOT SINK
| | in pipelines: 22(GETNEXT), 21(OPEN)
| |
| 21:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(80 AS INT), ss_quantity >= CAST(61 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -192,10 +192,10 @@ PLAN-ROOT SINK
| | in pipelines: 20(GETNEXT), 19(OPEN)
| |
| 19:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(80 AS INT), ss_quantity >= CAST(61 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -217,10 +217,10 @@ PLAN-ROOT SINK
| | in pipelines: 18(GETNEXT), 17(OPEN)
| |
| 17:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(60 AS INT), ss_quantity >= CAST(41 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -242,10 +242,10 @@ PLAN-ROOT SINK
| | in pipelines: 16(GETNEXT), 15(OPEN)
| |
| 15:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(60 AS INT), ss_quantity >= CAST(41 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -267,10 +267,10 @@ PLAN-ROOT SINK
| | in pipelines: 14(GETNEXT), 13(OPEN)
| |
| 13:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(60 AS INT), ss_quantity >= CAST(41 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -292,10 +292,10 @@ PLAN-ROOT SINK
| | in pipelines: 12(GETNEXT), 11(OPEN)
| |
| 11:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(40 AS INT), ss_quantity >= CAST(21 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -317,10 +317,10 @@ PLAN-ROOT SINK
| | in pipelines: 10(GETNEXT), 09(OPEN)
| |
| 09:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(40 AS INT), ss_quantity >= CAST(21 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -342,10 +342,10 @@ PLAN-ROOT SINK
| | in pipelines: 08(GETNEXT), 07(OPEN)
| |
| 07:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(40 AS INT), ss_quantity >= CAST(21 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -367,10 +367,10 @@ PLAN-ROOT SINK
| | in pipelines: 06(GETNEXT), 05(OPEN)
| |
| 05:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(20 AS INT), ss_quantity >= CAST(1 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -392,10 +392,10 @@ PLAN-ROOT SINK
| | in pipelines: 04(GETNEXT), 03(OPEN)
| |
| 03:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(20 AS INT), ss_quantity >= CAST(1 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -430,10 +430,10 @@ PLAN-ROOT SINK
| in pipelines: 02(GETNEXT), 01(OPEN)
|
01:SCAN HDFS [tpcds_parquet.store_sales]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
predicates: ss_quantity <= CAST(20 AS INT), ss_quantity >= CAST(1 AS INT)
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -483,10 +483,10 @@ PLAN-ROOT SINK
| | in pipelines: 30(GETNEXT), 29(OPEN)
| |
| 29:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(100 AS INT), ss_quantity >= CAST(81 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -528,10 +528,10 @@ PLAN-ROOT SINK
| | in pipelines: 28(GETNEXT), 27(OPEN)
| |
| 27:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(100 AS INT), ss_quantity >= CAST(81 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -573,10 +573,10 @@ PLAN-ROOT SINK
| | in pipelines: 26(GETNEXT), 25(OPEN)
| |
| 25:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(100 AS INT), ss_quantity >= CAST(81 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -618,10 +618,10 @@ PLAN-ROOT SINK
| | in pipelines: 24(GETNEXT), 23(OPEN)
| |
| 23:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(80 AS INT), ss_quantity >= CAST(61 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -663,10 +663,10 @@ PLAN-ROOT SINK
| | in pipelines: 22(GETNEXT), 21(OPEN)
| |
| 21:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(80 AS INT), ss_quantity >= CAST(61 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -708,10 +708,10 @@ PLAN-ROOT SINK
| | in pipelines: 20(GETNEXT), 19(OPEN)
| |
| 19:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(80 AS INT), ss_quantity >= CAST(61 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -753,10 +753,10 @@ PLAN-ROOT SINK
| | in pipelines: 18(GETNEXT), 17(OPEN)
| |
| 17:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(60 AS INT), ss_quantity >= CAST(41 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -798,10 +798,10 @@ PLAN-ROOT SINK
| | in pipelines: 16(GETNEXT), 15(OPEN)
| |
| 15:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(60 AS INT), ss_quantity >= CAST(41 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -843,10 +843,10 @@ PLAN-ROOT SINK
| | in pipelines: 14(GETNEXT), 13(OPEN)
| |
| 13:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(60 AS INT), ss_quantity >= CAST(41 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -888,10 +888,10 @@ PLAN-ROOT SINK
| | in pipelines: 12(GETNEXT), 11(OPEN)
| |
| 11:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(40 AS INT), ss_quantity >= CAST(21 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -933,10 +933,10 @@ PLAN-ROOT SINK
| | in pipelines: 10(GETNEXT), 09(OPEN)
| |
| 09:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(40 AS INT), ss_quantity >= CAST(21 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -978,10 +978,10 @@ PLAN-ROOT SINK
| | in pipelines: 08(GETNEXT), 07(OPEN)
| |
| 07:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(40 AS INT), ss_quantity >= CAST(21 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1023,10 +1023,10 @@ PLAN-ROOT SINK
| | in pipelines: 06(GETNEXT), 05(OPEN)
| |
| 05:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(20 AS INT), ss_quantity >= CAST(1 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1068,10 +1068,10 @@ PLAN-ROOT SINK
| | in pipelines: 04(GETNEXT), 03(OPEN)
| |
| 03:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(20 AS INT), ss_quantity >= CAST(1 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1126,10 +1126,10 @@ Per-Host Resources: mem-estimate=26.00MB mem-reservation=128.00KB thread-reserva
| in pipelines: 02(GETNEXT), 01(OPEN)
|
01:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
predicates: ss_quantity <= CAST(20 AS INT), ss_quantity >= CAST(1 AS INT)
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1186,10 +1186,10 @@ PLAN-ROOT SINK
| | in pipelines: 30(GETNEXT), 29(OPEN)
| |
| 29:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(100 AS INT), ss_quantity >= CAST(81 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1238,10 +1238,10 @@ PLAN-ROOT SINK
| | in pipelines: 28(GETNEXT), 27(OPEN)
| |
| 27:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(100 AS INT), ss_quantity >= CAST(81 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1290,10 +1290,10 @@ PLAN-ROOT SINK
| | in pipelines: 26(GETNEXT), 25(OPEN)
| |
| 25:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(100 AS INT), ss_quantity >= CAST(81 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1342,10 +1342,10 @@ PLAN-ROOT SINK
| | in pipelines: 24(GETNEXT), 23(OPEN)
| |
| 23:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(80 AS INT), ss_quantity >= CAST(61 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1394,10 +1394,10 @@ PLAN-ROOT SINK
| | in pipelines: 22(GETNEXT), 21(OPEN)
| |
| 21:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(80 AS INT), ss_quantity >= CAST(61 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1446,10 +1446,10 @@ PLAN-ROOT SINK
| | in pipelines: 20(GETNEXT), 19(OPEN)
| |
| 19:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(80 AS INT), ss_quantity >= CAST(61 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1498,10 +1498,10 @@ PLAN-ROOT SINK
| | in pipelines: 18(GETNEXT), 17(OPEN)
| |
| 17:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(60 AS INT), ss_quantity >= CAST(41 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1550,10 +1550,10 @@ PLAN-ROOT SINK
| | in pipelines: 16(GETNEXT), 15(OPEN)
| |
| 15:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(60 AS INT), ss_quantity >= CAST(41 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1602,10 +1602,10 @@ PLAN-ROOT SINK
| | in pipelines: 14(GETNEXT), 13(OPEN)
| |
| 13:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(60 AS INT), ss_quantity >= CAST(41 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1654,10 +1654,10 @@ PLAN-ROOT SINK
| | in pipelines: 12(GETNEXT), 11(OPEN)
| |
| 11:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(40 AS INT), ss_quantity >= CAST(21 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1706,10 +1706,10 @@ PLAN-ROOT SINK
| | in pipelines: 10(GETNEXT), 09(OPEN)
| |
| 09:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(40 AS INT), ss_quantity >= CAST(21 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1758,10 +1758,10 @@ PLAN-ROOT SINK
| | in pipelines: 08(GETNEXT), 07(OPEN)
| |
| 07:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(40 AS INT), ss_quantity >= CAST(21 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1810,10 +1810,10 @@ PLAN-ROOT SINK
| | in pipelines: 06(GETNEXT), 05(OPEN)
| |
| 05:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(20 AS INT), ss_quantity >= CAST(1 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1862,10 +1862,10 @@ PLAN-ROOT SINK
| | in pipelines: 04(GETNEXT), 03(OPEN)
| |
| 03:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_quantity <= CAST(20 AS INT), ss_quantity >= CAST(1 AS INT)
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1927,10 +1927,10 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=128.00KB thread-res
| in pipelines: 02(GETNEXT), 01(OPEN)
|
01:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
predicates: ss_quantity <= CAST(20 AS INT), ss_quantity >= CAST(1 AS INT)
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q10a.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q10a.test
index 39dc738..493fd48 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q10a.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q10a.test
@@ -82,14 +82,14 @@ PLAN-ROOT SINK
|
16:HASH JOIN [RIGHT SEMI JOIN]
| hash predicates: customer_sk = c.c_customer_sk
-| runtime filters: RF000[bloom] <- c.c_customer_sk
+| runtime filters: RF000[bloom] <- c.c_customer_sk, RF001[min_max] <- c.c_customer_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,0,1 row-size=129B cardinality=318
| in pipelines: 07(GETNEXT), 10(GETNEXT), 03(OPEN)
|
|--15:HASH JOIN [RIGHT SEMI JOIN]
| | hash predicates: ss_customer_sk = c.c_customer_sk
-| | runtime filters: RF006[bloom] <- c.c_customer_sk
+| | runtime filters: RF006[bloom] <- c.c_customer_sk, RF007[min_max] <- c.c_customer_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=2,0,1 row-size=129B cardinality=318
| | in pipelines: 03(GETNEXT), 02(OPEN)
@@ -97,7 +97,7 @@ PLAN-ROOT SINK
| |--14:HASH JOIN [INNER JOIN]
| | | hash predicates: c.c_current_addr_sk = ca.ca_address_sk
| | | fk/pk conjuncts: c.c_current_addr_sk = ca.ca_address_sk
-| | | runtime filters: RF010[bloom] <- ca.ca_address_sk
+| | | runtime filters: RF010[bloom] <- ca.ca_address_sk, RF011[min_max] <- ca.ca_address_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=2,0,1 row-size=129B cardinality=318
| | | in pipelines: 02(GETNEXT), 01(OPEN)
@@ -118,14 +118,14 @@ PLAN-ROOT SINK
| | 13:HASH JOIN [INNER JOIN]
| | | hash predicates: cd_demo_sk = c.c_current_cdemo_sk
| | | fk/pk conjuncts: none
-| | | runtime filters: RF012[bloom] <- c.c_current_cdemo_sk
+| | | runtime filters: RF012[bloom] <- c.c_current_cdemo_sk, RF013[min_max] <- c.c_current_cdemo_sk
| | | mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
| | | tuple-ids=2,0 row-size=99B cardinality=100.00K
| | | in pipelines: 02(GETNEXT), 00(OPEN)
| | |
| | |--00:SCAN HDFS [tpcds_parquet.customer c]
| | | HDFS partitions=1/1 files=1 size=5.49MB
-| | | runtime filters: RF010[bloom] -> c.c_current_addr_sk
+| | | runtime filters: RF011[min_max] -> c.c_current_addr_sk, RF010[bloom] -> c.c_current_addr_sk
| | | stored statistics:
| | | table: rows=100.00K size=5.49MB
| | | columns: all
@@ -136,7 +136,7 @@ PLAN-ROOT SINK
| | |
| | 02:SCAN HDFS [tpcds_parquet.customer_demographics]
| | HDFS partitions=1/1 files=1 size=7.49MB
-| | runtime filters: RF012[bloom] -> cd_demo_sk
+| | runtime filters: RF013[min_max] -> cd_demo_sk, RF012[bloom] -> cd_demo_sk
| | stored statistics:
| | table: rows=1.92M size=7.49MB
| | columns: all
@@ -167,10 +167,10 @@ PLAN-ROOT SINK
| | in pipelines: 04(GETNEXT)
| |
| 03:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
-| runtime filters: RF006[bloom] -> ss_customer_sk, RF008[bloom] -> ss_sold_date_sk
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
+| runtime filters: RF007[min_max] -> ss_customer_sk, RF006[bloom] -> ss_customer_sk, RF008[bloom] -> ss_sold_date_sk
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -186,7 +186,7 @@ PLAN-ROOT SINK
|--12:HASH JOIN [INNER JOIN]
| | hash predicates: cs_sold_date_sk = d_date_sk
| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
-| | runtime filters: RF004[bloom] <- d_date_sk
+| | runtime filters: RF004[bloom] <- d_date_sk, RF005[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=8,9 row-size=20B cardinality=93.21K
| | in pipelines: 10(GETNEXT), 11(OPEN)
@@ -206,7 +206,7 @@ PLAN-ROOT SINK
| |
| 10:SCAN HDFS [tpcds_parquet.catalog_sales]
| HDFS partitions=1/1 files=3 size=96.62MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.catalog_sales.cs_ship_customer_sk, RF004[bloom] -> cs_sold_date_sk
+| runtime filters: RF001[min_max] -> tpcds_parquet.catalog_sales.cs_ship_customer_sk, RF005[min_max] -> cs_sold_date_sk, RF000[bloom] -> tpcds_parquet.catalog_sales.cs_ship_customer_sk, RF004[bloom] -> cs_sold_date_sk
| stored statistics:
| table: rows=1.44M size=96.62MB
| columns: all
@@ -218,7 +218,7 @@ PLAN-ROOT SINK
09:HASH JOIN [INNER JOIN]
| hash predicates: ws_sold_date_sk = d_date_sk
| fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| runtime filters: RF002[bloom] <- d_date_sk
+| runtime filters: RF002[bloom] <- d_date_sk, RF003[min_max] <- d_date_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=6,7 row-size=20B cardinality=46.82K
| in pipelines: 07(GETNEXT), 08(OPEN)
@@ -238,7 +238,7 @@ PLAN-ROOT SINK
|
07:SCAN HDFS [tpcds_parquet.web_sales]
HDFS partitions=1/1 files=2 size=45.09MB
- runtime filters: RF000[bloom] -> tpcds_parquet.web_sales.ws_bill_customer_sk, RF002[bloom] -> ws_sold_date_sk
+ runtime filters: RF001[min_max] -> tpcds_parquet.web_sales.ws_bill_customer_sk, RF003[min_max] -> ws_sold_date_sk, RF000[bloom] -> tpcds_parquet.web_sales.ws_bill_customer_sk, RF002[bloom] -> ws_sold_date_sk
stored statistics:
table: rows=719.38K size=45.09MB
columns: all
@@ -293,14 +293,14 @@ Per-Host Resources: mem-estimate=17.34MB mem-reservation=7.88MB thread-reservati
|
16:HASH JOIN [RIGHT SEMI JOIN, PARTITIONED]
| hash predicates: customer_sk = c.c_customer_sk
-| runtime filters: RF000[bloom] <- c.c_customer_sk
+| runtime filters: RF000[bloom] <- c.c_customer_sk, RF001[min_max] <- c.c_customer_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,0,1 row-size=129B cardinality=318
| in pipelines: 07(GETNEXT), 10(GETNEXT), 03(OPEN)
|
|--15:HASH JOIN [RIGHT SEMI JOIN, PARTITIONED]
| | hash predicates: ss_customer_sk = c.c_customer_sk
-| | runtime filters: RF006[bloom] <- c.c_customer_sk
+| | runtime filters: RF006[bloom] <- c.c_customer_sk, RF007[min_max] <- c.c_customer_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=2,0,1 row-size=129B cardinality=318
| | in pipelines: 03(GETNEXT), 02(OPEN)
@@ -315,7 +315,7 @@ Per-Host Resources: mem-estimate=17.34MB mem-reservation=7.88MB thread-reservati
| | 14:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash predicates: c.c_current_addr_sk = ca.ca_address_sk
| | | fk/pk conjuncts: c.c_current_addr_sk = ca.ca_address_sk
-| | | runtime filters: RF010[bloom] <- ca.ca_address_sk
+| | | runtime filters: RF010[bloom] <- ca.ca_address_sk, RF011[min_max] <- ca.ca_address_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=2,0,1 row-size=129B cardinality=318
| | | in pipelines: 02(GETNEXT), 01(OPEN)
@@ -343,7 +343,7 @@ Per-Host Resources: mem-estimate=17.34MB mem-reservation=7.88MB thread-reservati
| | 13:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash predicates: cd_demo_sk = c.c_current_cdemo_sk
| | | fk/pk conjuncts: none
-| | | runtime filters: RF012[bloom] <- c.c_current_cdemo_sk
+| | | runtime filters: RF012[bloom] <- c.c_current_cdemo_sk, RF013[min_max] <- c.c_current_cdemo_sk
| | | mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
| | | tuple-ids=2,0 row-size=99B cardinality=100.00K
| | | in pipelines: 02(GETNEXT), 00(OPEN)
@@ -357,7 +357,7 @@ Per-Host Resources: mem-estimate=17.34MB mem-reservation=7.88MB thread-reservati
| | | Per-Host Resources: mem-estimate=49.00MB mem-reservation=3.00MB thread-reservation=2 runtime-filters-memory=1.00MB
| | | 00:SCAN HDFS [tpcds_parquet.customer c, RANDOM]
| | | HDFS partitions=1/1 files=1 size=5.49MB
-| | | runtime filters: RF010[bloom] -> c.c_current_addr_sk
+| | | runtime filters: RF011[min_max] -> c.c_current_addr_sk, RF010[bloom] -> c.c_current_addr_sk
| | | stored statistics:
| | | table: rows=100.00K size=5.49MB
| | | columns: all
@@ -368,7 +368,7 @@ Per-Host Resources: mem-estimate=17.34MB mem-reservation=7.88MB thread-reservati
| | |
| | 02:SCAN HDFS [tpcds_parquet.customer_demographics, RANDOM]
| | HDFS partitions=1/1 files=1 size=7.49MB
-| | runtime filters: RF012[bloom] -> cd_demo_sk
+| | runtime filters: RF013[min_max] -> cd_demo_sk, RF012[bloom] -> cd_demo_sk
| | stored statistics:
| | table: rows=1.92M size=7.49MB
| | columns: all
@@ -413,10 +413,10 @@ Per-Host Resources: mem-estimate=17.34MB mem-reservation=7.88MB thread-reservati
| | in pipelines: 04(GETNEXT)
| |
| 03:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
-| runtime filters: RF006[bloom] -> ss_customer_sk, RF008[bloom] -> ss_sold_date_sk
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
+| runtime filters: RF007[min_max] -> ss_customer_sk, RF006[bloom] -> ss_customer_sk, RF008[bloom] -> ss_sold_date_sk
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -439,7 +439,7 @@ Per-Host Resources: mem-estimate=100.95MB mem-reservation=8.94MB thread-reservat
|--12:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: cs_sold_date_sk = d_date_sk
| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
-| | runtime filters: RF004[bloom] <- d_date_sk
+| | runtime filters: RF004[bloom] <- d_date_sk, RF005[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=8,9 row-size=20B cardinality=93.21K
| | in pipelines: 10(GETNEXT), 11(OPEN)
@@ -466,7 +466,7 @@ Per-Host Resources: mem-estimate=100.95MB mem-reservation=8.94MB thread-reservat
| |
| 10:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| HDFS partitions=1/1 files=3 size=96.62MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.catalog_sales.cs_ship_customer_sk, RF004[bloom] -> cs_sold_date_sk
+| runtime filters: RF001[min_max] -> tpcds_parquet.catalog_sales.cs_ship_customer_sk, RF005[min_max] -> cs_sold_date_sk, RF000[bloom] -> tpcds_parquet.catalog_sales.cs_ship_customer_sk, RF004[bloom] -> cs_sold_date_sk
| stored statistics:
| table: rows=1.44M size=96.62MB
| columns: all
@@ -478,7 +478,7 @@ Per-Host Resources: mem-estimate=100.95MB mem-reservation=8.94MB thread-reservat
09:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: ws_sold_date_sk = d_date_sk
| fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| runtime filters: RF002[bloom] <- d_date_sk
+| runtime filters: RF002[bloom] <- d_date_sk, RF003[min_max] <- d_date_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=6,7 row-size=20B cardinality=46.82K
| in pipelines: 07(GETNEXT), 08(OPEN)
@@ -505,7 +505,7 @@ Per-Host Resources: mem-estimate=100.95MB mem-reservation=8.94MB thread-reservat
|
07:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
HDFS partitions=1/1 files=2 size=45.09MB
- runtime filters: RF000[bloom] -> tpcds_parquet.web_sales.ws_bill_customer_sk, RF002[bloom] -> ws_sold_date_sk
+ runtime filters: RF001[min_max] -> tpcds_parquet.web_sales.ws_bill_customer_sk, RF003[min_max] -> ws_sold_date_sk, RF000[bloom] -> tpcds_parquet.web_sales.ws_bill_customer_sk, RF002[bloom] -> ws_sold_date_sk
stored statistics:
table: rows=719.38K size=45.09MB
columns: all
@@ -570,7 +570,7 @@ Per-Instance Resources: mem-estimate=10.20MB mem-reservation=2.00MB thread-reser
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: c.c_customer_sk
-| | runtime filters: RF000[bloom] <- c.c_customer_sk
+| | runtime filters: RF000[bloom] <- c.c_customer_sk, RF001[min_max] <- c.c_customer_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| |
| 15:HASH JOIN [RIGHT SEMI JOIN, PARTITIONED]
@@ -585,7 +585,7 @@ Per-Instance Resources: mem-estimate=10.20MB mem-reservation=2.00MB thread-reser
| | JOIN BUILD
| | | join-table-id=01 plan-id=02 cohort-id=02
| | | build expressions: c.c_customer_sk
-| | | runtime filters: RF006[bloom] <- c.c_customer_sk
+| | | runtime filters: RF006[bloom] <- c.c_customer_sk, RF007[min_max] <- c.c_customer_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | |
| | 25:EXCHANGE [HASH(c.c_customer_sk)]
@@ -609,7 +609,7 @@ Per-Instance Resources: mem-estimate=10.20MB mem-reservation=2.00MB thread-reser
| | | JOIN BUILD
| | | | join-table-id=02 plan-id=03 cohort-id=03
| | | | build expressions: ca.ca_address_sk
-| | | | runtime filters: RF010[bloom] <- ca.ca_address_sk
+| | | | runtime filters: RF010[bloom] <- ca.ca_address_sk, RF011[min_max] <- ca.ca_address_sk
| | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | |
| | | 23:EXCHANGE [BROADCAST]
@@ -645,7 +645,7 @@ Per-Instance Resources: mem-estimate=10.20MB mem-reservation=2.00MB thread-reser
| | | JOIN BUILD
| | | | join-table-id=03 plan-id=04 cohort-id=03
| | | | build expressions: c.c_current_cdemo_sk
-| | | | runtime filters: RF012[bloom] <- c.c_current_cdemo_sk
+| | | | runtime filters: RF012[bloom] <- c.c_current_cdemo_sk, RF013[min_max] <- c.c_current_cdemo_sk
| | | | mem-estimate=5.75MB mem-reservation=5.75MB spill-buffer=128.00KB thread-reservation=0
| | | |
| | | 22:EXCHANGE [BROADCAST]
@@ -658,7 +658,7 @@ Per-Instance Resources: mem-estimate=10.20MB mem-reservation=2.00MB thread-reser
| | | Per-Instance Resources: mem-estimate=16.00MB mem-reservation=2.00MB thread-reservation=1
| | | 00:SCAN HDFS [tpcds_parquet.customer c, RANDOM]
| | | HDFS partitions=1/1 files=1 size=5.49MB
-| | | runtime filters: RF010[bloom] -> c.c_current_addr_sk
+| | | runtime filters: RF011[min_max] -> c.c_current_addr_sk, RF010[bloom] -> c.c_current_addr_sk
| | | stored statistics:
| | | table: rows=100.00K size=5.49MB
| | | columns: all
@@ -669,7 +669,7 @@ Per-Instance Resources: mem-estimate=10.20MB mem-reservation=2.00MB thread-reser
| | |
| | 02:SCAN HDFS [tpcds_parquet.customer_demographics, RANDOM]
| | HDFS partitions=1/1 files=1 size=7.49MB
-| | runtime filters: RF012[bloom] -> cd_demo_sk
+| | runtime filters: RF013[min_max] -> cd_demo_sk, RF012[bloom] -> cd_demo_sk
| | stored statistics:
| | table: rows=1.92M size=7.49MB
| | columns: all
@@ -723,10 +723,10 @@ Per-Instance Resources: mem-estimate=10.20MB mem-reservation=2.00MB thread-reser
| | in pipelines: 04(GETNEXT)
| |
| 03:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
-| runtime filters: RF006[bloom] -> ss_customer_sk, RF008[bloom] -> ss_sold_date_sk
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
+| runtime filters: RF007[min_max] -> ss_customer_sk, RF006[bloom] -> ss_customer_sk, RF008[bloom] -> ss_sold_date_sk
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -760,7 +760,7 @@ Per-Instance Resources: mem-estimate=48.00MB mem-reservation=4.00MB thread-reser
| | JOIN BUILD
| | | join-table-id=06 plan-id=07 cohort-id=01
| | | build expressions: d_date_sk
-| | | runtime filters: RF004[bloom] <- d_date_sk
+| | | runtime filters: RF004[bloom] <- d_date_sk, RF005[min_max] <- d_date_sk
| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | |
| | 20:EXCHANGE [BROADCAST]
@@ -785,7 +785,7 @@ Per-Instance Resources: mem-estimate=48.00MB mem-reservation=4.00MB thread-reser
| |
| 10:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| HDFS partitions=1/1 files=3 size=96.62MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.catalog_sales.cs_ship_customer_sk, RF004[bloom] -> cs_sold_date_sk
+| runtime filters: RF001[min_max] -> tpcds_parquet.catalog_sales.cs_ship_customer_sk, RF005[min_max] -> cs_sold_date_sk, RF000[bloom] -> tpcds_parquet.catalog_sales.cs_ship_customer_sk, RF004[bloom] -> cs_sold_date_sk
| stored statistics:
| table: rows=1.44M size=96.62MB
| columns: all
@@ -807,7 +807,7 @@ Per-Instance Resources: mem-estimate=48.00MB mem-reservation=4.00MB thread-reser
| JOIN BUILD
| | join-table-id=05 plan-id=06 cohort-id=01
| | build expressions: d_date_sk
-| | runtime filters: RF002[bloom] <- d_date_sk
+| | runtime filters: RF002[bloom] <- d_date_sk, RF003[min_max] <- d_date_sk
| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| |
| 19:EXCHANGE [BROADCAST]
@@ -832,7 +832,7 @@ Per-Instance Resources: mem-estimate=48.00MB mem-reservation=4.00MB thread-reser
|
07:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
HDFS partitions=1/1 files=2 size=45.09MB
- runtime filters: RF000[bloom] -> tpcds_parquet.web_sales.ws_bill_customer_sk, RF002[bloom] -> ws_sold_date_sk
+ runtime filters: RF001[min_max] -> tpcds_parquet.web_sales.ws_bill_customer_sk, RF003[min_max] -> ws_sold_date_sk, RF000[bloom] -> tpcds_parquet.web_sales.ws_bill_customer_sk, RF002[bloom] -> ws_sold_date_sk
stored statistics:
table: rows=719.38K size=45.09MB
columns: all
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q11.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q11.test
index 01dc63e..7f4cba1 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q11.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q11.test
@@ -79,7 +79,6 @@ ORDER BY t_s_secyear.customer_id,
t_s_secyear.customer_last_name,
t_s_secyear.customer_preferred_cust_flag
LIMIT 100;
-
---- PLAN
Max Per-Host Resource Reservation: Memory=265.06MB Threads=13
Per-Host Resource Estimates: Memory=1.11GB
@@ -99,7 +98,7 @@ PLAN-ROOT SINK
| hash predicates: customer_id = customer_id
| fk/pk conjuncts: assumed fk/pk
| other predicates: CASE WHEN year_total > CAST(0 AS DECIMAL(3,0)) THEN (year_total * CAST(1.0000 AS DECIMAL(5,4))) / year_total ELSE CAST(0 AS DECIMAL(38,6)) END > CASE WHEN year_total > CAST(0 AS DECIMAL(3,0)) THEN (year_total * CAST(1.0000 AS DECIMAL(5,4))) / year_total ELSE CAST(0 AS DECIMAL(38,6)) END
-| runtime filters: RF000[bloom] <- customer_id
+| runtime filters: RF000[bloom] <- customer_id, RF001[min_max] <- customer_id
| mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| tuple-ids=18,8,28,38 row-size=225B cardinality=589.03K
| in pipelines: 13(GETNEXT), 27(OPEN)
@@ -119,7 +118,7 @@ PLAN-ROOT SINK
| 26:HASH JOIN [INNER JOIN]
| | hash predicates: ws_sold_date_sk = d_date_sk
| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| | runtime filters: RF018[bloom] <- d_date_sk
+| | runtime filters: RF018[bloom] <- d_date_sk, RF019[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=34,35,36 row-size=177B cardinality=148.00K
| | in pipelines: 22(GETNEXT), 24(OPEN)
@@ -140,13 +139,14 @@ PLAN-ROOT SINK
| 25:HASH JOIN [INNER JOIN]
| | hash predicates: c_customer_sk = ws_bill_customer_sk
| | fk/pk conjuncts: none
+| | runtime filters: RF021[min_max] <- ws_bill_customer_sk
| | mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB thread-reservation=0
| | tuple-ids=34,35 row-size=169B cardinality=719.38K
| | in pipelines: 22(GETNEXT), 23(OPEN)
| |
| |--23:SCAN HDFS [tpcds_parquet.web_sales]
| | HDFS partitions=1/1 files=2 size=45.09MB
-| | runtime filters: RF018[bloom] -> ws_sold_date_sk
+| | runtime filters: RF019[min_max] -> ws_sold_date_sk, RF018[bloom] -> ws_sold_date_sk
| | stored statistics:
| | table: rows=719.38K size=45.09MB
| | columns: all
@@ -157,6 +157,7 @@ PLAN-ROOT SINK
| |
| 22:SCAN HDFS [tpcds_parquet.customer]
| HDFS partitions=1/1 files=1 size=5.49MB
+| runtime filters: RF021[min_max] -> c_customer_sk
| stored statistics:
| table: rows=100.00K size=5.49MB
| columns: all
@@ -168,7 +169,7 @@ PLAN-ROOT SINK
29:HASH JOIN [INNER JOIN]
| hash predicates: customer_id = customer_id
| fk/pk conjuncts: assumed fk/pk
-| runtime filters: RF002[bloom] <- customer_id
+| runtime filters: RF002[bloom] <- customer_id, RF003[min_max] <- customer_id
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=18,8,28 row-size=181B cardinality=589.03K
| in pipelines: 13(GETNEXT), 20(OPEN)
@@ -189,7 +190,7 @@ PLAN-ROOT SINK
| 19:HASH JOIN [INNER JOIN]
| | hash predicates: ws_sold_date_sk = d_date_sk
| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| | runtime filters: RF014[bloom] <- d_date_sk
+| | runtime filters: RF014[bloom] <- d_date_sk, RF015[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=24,25,26 row-size=177B cardinality=148.00K
| | in pipelines: 15(GETNEXT), 17(OPEN)
@@ -210,14 +211,14 @@ PLAN-ROOT SINK
| 18:HASH JOIN [INNER JOIN]
| | hash predicates: c_customer_sk = ws_bill_customer_sk
| | fk/pk conjuncts: none
-| | runtime filters: RF016[bloom] <- ws_bill_customer_sk
+| | runtime filters: RF016[bloom] <- ws_bill_customer_sk, RF017[min_max] <- ws_bill_customer_sk
| | mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB thread-reservation=0
| | tuple-ids=24,25 row-size=169B cardinality=719.38K
| | in pipelines: 15(GETNEXT), 16(OPEN)
| |
| |--16:SCAN HDFS [tpcds_parquet.web_sales]
| | HDFS partitions=1/1 files=2 size=45.09MB
-| | runtime filters: RF014[bloom] -> ws_sold_date_sk
+| | runtime filters: RF015[min_max] -> ws_sold_date_sk, RF014[bloom] -> ws_sold_date_sk
| | stored statistics:
| | table: rows=719.38K size=45.09MB
| | columns: all
@@ -228,7 +229,7 @@ PLAN-ROOT SINK
| |
| 15:SCAN HDFS [tpcds_parquet.customer]
| HDFS partitions=1/1 files=1 size=5.49MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF016[bloom] -> c_customer_sk
+| runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF017[min_max] -> c_customer_sk, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF016[bloom] -> c_customer_sk
| stored statistics:
| table: rows=100.00K size=5.49MB
| columns: all
@@ -240,7 +241,7 @@ PLAN-ROOT SINK
28:HASH JOIN [INNER JOIN]
| hash predicates: customer_id = customer_id
| fk/pk conjuncts: assumed fk/pk
-| runtime filters: RF004[bloom] <- customer_id
+| runtime filters: RF004[bloom] <- customer_id, RF005[min_max] <- customer_id
| mem-estimate=4.75MB mem-reservation=4.75MB spill-buffer=256.00KB thread-reservation=0
| tuple-ids=18,8 row-size=137B cardinality=589.03K
| in pipelines: 13(GETNEXT), 06(OPEN)
@@ -261,7 +262,7 @@ PLAN-ROOT SINK
| 05:HASH JOIN [INNER JOIN]
| | hash predicates: c_customer_sk = ss_customer_sk
| | fk/pk conjuncts: c_customer_sk = ss_customer_sk
-| | runtime filters: RF010[bloom] <- ss_customer_sk
+| | runtime filters: RF010[bloom] <- ss_customer_sk, RF011[min_max] <- ss_customer_sk
| | mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB thread-reservation=0
| | tuple-ids=0,1,2 row-size=177B cardinality=589.03K
| | in pipelines: 01(GETNEXT), 02(OPEN)
@@ -288,10 +289,10 @@ PLAN-ROOT SINK
| | | in pipelines: 03(GETNEXT)
| | |
| | 02:SCAN HDFS [tpcds_parquet.store_sales]
-| | HDFS partitions=1824/1824 files=1824 size=200.95MB
+| | HDFS partitions=1824/1824 files=1824 size=200.96MB
| | runtime filters: RF012[bloom] -> ss_sold_date_sk
| | stored statistics:
-| | table: rows=2.88M size=200.95MB
+| | table: rows=2.88M size=200.96MB
| | partitions: 1824/1824 rows=2.88M
| | columns: all
| | extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -301,7 +302,7 @@ PLAN-ROOT SINK
| |
| 01:SCAN HDFS [tpcds_parquet.customer]
| HDFS partitions=1/1 files=1 size=5.49MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF010[bloom] -> c_customer_sk
+| runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF011[min_max] -> c_customer_sk, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF010[bloom] -> c_customer_sk
| stored statistics:
| table: rows=100.00K size=5.49MB
| columns: all
@@ -325,7 +326,7 @@ PLAN-ROOT SINK
12:HASH JOIN [INNER JOIN]
| hash predicates: c_customer_sk = ss_customer_sk
| fk/pk conjuncts: c_customer_sk = ss_customer_sk
-| runtime filters: RF006[bloom] <- ss_customer_sk
+| runtime filters: RF006[bloom] <- ss_customer_sk, RF007[min_max] <- ss_customer_sk
| mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=1.00MB thread-reservation=0
| tuple-ids=10,11,12 row-size=177B cardinality=589.03K
| in pipelines: 08(GETNEXT), 09(OPEN)
@@ -352,10 +353,10 @@ PLAN-ROOT SINK
| | in pipelines: 10(GETNEXT)
| |
| 09:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| runtime filters: RF008[bloom] -> ss_sold_date_sk
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -365,7 +366,7 @@ PLAN-ROOT SINK
|
08:SCAN HDFS [tpcds_parquet.customer]
HDFS partitions=1/1 files=1 size=5.49MB
- runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id, RF006[bloom] -> c_customer_sk
+ runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF005[min_max] -> tpcds_parquet.customer.c_customer_id, RF007[min_max] -> c_customer_sk, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id, RF006[bloom] -> c_customer_sk
stored statistics:
table: rows=100.00K size=5.49MB
columns: all
@@ -401,7 +402,7 @@ Per-Host Resources: mem-estimate=90.58MB mem-reservation=52.19MB thread-reservat
| hash predicates: customer_id = customer_id
| fk/pk conjuncts: assumed fk/pk
| other predicates: CASE WHEN year_total > CAST(0 AS DECIMAL(3,0)) THEN (year_total * CAST(1.0000 AS DECIMAL(5,4))) / year_total ELSE CAST(0 AS DECIMAL(38,6)) END > CASE WHEN year_total > CAST(0 AS DECIMAL(3,0)) THEN (year_total * CAST(1.0000 AS DECIMAL(5,4))) / year_total ELSE CAST(0 AS DECIMAL(38,6)) END
-| runtime filters: RF000[bloom] <- customer_id
+| runtime filters: RF000[bloom] <- customer_id, RF001[min_max] <- customer_id
| mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| tuple-ids=18,8,28,38 row-size=225B cardinality=589.03K
| in pipelines: 36(GETNEXT), 53(OPEN)
@@ -442,7 +443,7 @@ Per-Host Resources: mem-estimate=90.58MB mem-reservation=52.19MB thread-reservat
| 26:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: ws_sold_date_sk = d_date_sk
| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| | runtime filters: RF018[bloom] <- d_date_sk
+| | runtime filters: RF018[bloom] <- d_date_sk, RF019[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=35,34,36 row-size=177B cardinality=148.00K
| | in pipelines: 23(GETNEXT), 24(OPEN)
@@ -470,6 +471,7 @@ Per-Host Resources: mem-estimate=90.58MB mem-reservation=52.19MB thread-reservat
| 25:HASH JOIN [INNER JOIN, PARTITIONED]
| | hash predicates: ws_bill_customer_sk = c_customer_sk
| | fk/pk conjuncts: none
+| | runtime filters: RF021[min_max] <- c_customer_sk
| | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| | tuple-ids=35,34 row-size=169B cardinality=719.38K
| | in pipelines: 23(GETNEXT), 22(OPEN)
@@ -500,7 +502,7 @@ Per-Host Resources: mem-estimate=90.58MB mem-reservation=52.19MB thread-reservat
| Per-Host Resources: mem-estimate=129.00MB mem-reservation=9.00MB thread-reservation=2 runtime-filters-memory=1.00MB
| 23:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| HDFS partitions=1/1 files=2 size=45.09MB
-| runtime filters: RF018[bloom] -> ws_sold_date_sk
+| runtime filters: RF019[min_max] -> ws_sold_date_sk, RF021[min_max] -> ws_bill_customer_sk, RF018[bloom] -> ws_sold_date_sk
| stored statistics:
| table: rows=719.38K size=45.09MB
| columns: all
@@ -512,7 +514,7 @@ Per-Host Resources: mem-estimate=90.58MB mem-reservation=52.19MB thread-reservat
29:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: customer_id = customer_id
| fk/pk conjuncts: assumed fk/pk
-| runtime filters: RF002[bloom] <- customer_id
+| runtime filters: RF002[bloom] <- customer_id, RF003[min_max] <- customer_id
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=18,8,28 row-size=181B cardinality=589.03K
| in pipelines: 36(GETNEXT), 47(OPEN)
@@ -554,7 +556,7 @@ Per-Host Resources: mem-estimate=90.58MB mem-reservation=52.19MB thread-reservat
| 19:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: ws_sold_date_sk = d_date_sk
| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| | runtime filters: RF014[bloom] <- d_date_sk
+| | runtime filters: RF014[bloom] <- d_date_sk, RF015[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=25,24,26 row-size=177B cardinality=148.00K
| | in pipelines: 16(GETNEXT), 17(OPEN)
@@ -582,7 +584,7 @@ Per-Host Resources: mem-estimate=90.58MB mem-reservation=52.19MB thread-reservat
| 18:HASH JOIN [INNER JOIN, PARTITIONED]
| | hash predicates: ws_bill_customer_sk = c_customer_sk
| | fk/pk conjuncts: none
-| | runtime filters: RF016[bloom] <- c_customer_sk
+| | runtime filters: RF016[bloom] <- c_customer_sk, RF017[min_max] <- c_customer_sk
| | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| | tuple-ids=25,24 row-size=169B cardinality=719.38K
| | in pipelines: 16(GETNEXT), 15(OPEN)
@@ -596,7 +598,7 @@ Per-Host Resources: mem-estimate=90.58MB mem-reservation=52.19MB thread-reservat
| | Per-Host Resources: mem-estimate=129.00MB mem-reservation=9.00MB thread-reservation=2 runtime-filters-memory=1.00MB
| | 15:SCAN HDFS [tpcds_parquet.customer, RANDOM]
| | HDFS partitions=1/1 files=1 size=5.49MB
-| | runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id
+| | runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id
| | stored statistics:
| | table: rows=100.00K size=5.49MB
| | columns: all
@@ -614,7 +616,7 @@ Per-Host Resources: mem-estimate=90.58MB mem-reservation=52.19MB thread-reservat
| Per-Host Resources: mem-estimate=130.00MB mem-reservation=10.00MB thread-reservation=2 runtime-filters-memory=2.00MB
| 16:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| HDFS partitions=1/1 files=2 size=45.09MB
-| runtime filters: RF014[bloom] -> ws_sold_date_sk, RF016[bloom] -> ws_bill_customer_sk
+| runtime filters: RF015[min_max] -> ws_sold_date_sk, RF017[min_max] -> ws_bill_customer_sk, RF014[bloom] -> ws_sold_date_sk, RF016[bloom] -> ws_bill_customer_sk
| stored statistics:
| table: rows=719.38K size=45.09MB
| columns: all
@@ -626,7 +628,7 @@ Per-Host Resources: mem-estimate=90.58MB mem-reservation=52.19MB thread-reservat
28:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: customer_id = customer_id
| fk/pk conjuncts: assumed fk/pk
-| runtime filters: RF004[bloom] <- customer_id
+| runtime filters: RF004[bloom] <- customer_id, RF005[min_max] <- customer_id
| mem-estimate=4.75MB mem-reservation=4.75MB spill-buffer=256.00KB thread-reservation=0
| tuple-ids=18,8 row-size=137B cardinality=589.03K
| in pipelines: 36(GETNEXT), 41(OPEN)
@@ -668,7 +670,7 @@ Per-Host Resources: mem-estimate=90.58MB mem-reservation=52.19MB thread-reservat
| 05:HASH JOIN [INNER JOIN, PARTITIONED]
| | hash predicates: ss_customer_sk = c_customer_sk
| | fk/pk conjuncts: ss_customer_sk = c_customer_sk
-| | runtime filters: RF010[bloom] <- c_customer_sk
+| | runtime filters: RF010[bloom] <- c_customer_sk, RF011[min_max] <- c_customer_sk
| | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| | tuple-ids=1,2,0 row-size=177B cardinality=589.03K
| | in pipelines: 02(GETNEXT), 01(OPEN)
@@ -682,7 +684,7 @@ Per-Host Resources: mem-estimate=90.58MB mem-reservation=52.19MB thread-reservat
| | Per-Host Resources: mem-estimate=130.00MB mem-reservation=10.00MB thread-reservation=2 runtime-filters-memory=2.00MB
| | 01:SCAN HDFS [tpcds_parquet.customer, RANDOM]
| | HDFS partitions=1/1 files=1 size=5.49MB
-| | runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id
+| | runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id
| | stored statistics:
| | table: rows=100.00K size=5.49MB
| | columns: all
@@ -727,10 +729,10 @@ Per-Host Resources: mem-estimate=90.58MB mem-reservation=52.19MB thread-reservat
| | in pipelines: 03(GETNEXT)
| |
| 02:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
-| runtime filters: RF012[bloom] -> ss_sold_date_sk, RF010[bloom] -> ss_customer_sk
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
+| runtime filters: RF011[min_max] -> ss_customer_sk, RF012[bloom] -> ss_sold_date_sk, RF010[bloom] -> ss_customer_sk
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -767,7 +769,7 @@ Per-Host Resources: mem-estimate=76.49MB mem-reservation=43.50MB thread-reservat
12:HASH JOIN [INNER JOIN, PARTITIONED]
| hash predicates: ss_customer_sk = c_customer_sk
| fk/pk conjuncts: ss_customer_sk = c_customer_sk
-| runtime filters: RF006[bloom] <- c_customer_sk
+| runtime filters: RF006[bloom] <- c_customer_sk, RF007[min_max] <- c_customer_sk
| mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| tuple-ids=11,12,10 row-size=177B cardinality=589.03K
| in pipelines: 09(GETNEXT), 08(OPEN)
@@ -781,7 +783,7 @@ Per-Host Resources: mem-estimate=76.49MB mem-reservation=43.50MB thread-reservat
| Per-Host Resources: mem-estimate=131.00MB mem-reservation=11.00MB thread-reservation=2 runtime-filters-memory=3.00MB
| 08:SCAN HDFS [tpcds_parquet.customer, RANDOM]
| HDFS partitions=1/1 files=1 size=5.49MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id
+| runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF005[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id
| stored statistics:
| table: rows=100.00K size=5.49MB
| columns: all
@@ -826,10 +828,10 @@ Per-Host Resources: mem-estimate=51.95MB mem-reservation=5.94MB thread-reservati
| in pipelines: 10(GETNEXT)
|
09:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF008[bloom] -> ss_sold_date_sk, RF006[bloom] -> ss_customer_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF007[min_max] -> ss_customer_sk, RF008[bloom] -> ss_sold_date_sk, RF006[bloom] -> ss_customer_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -874,7 +876,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: customer_id
-| | runtime filters: RF000[bloom] <- customer_id
+| | runtime filters: RF000[bloom] <- customer_id, RF001[min_max] <- customer_id
| | mem-estimate=17.00MB mem-reservation=17.00MB spill-buffer=512.00KB thread-reservation=0
| |
| 54:EXCHANGE [BROADCAST]
@@ -923,7 +925,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | JOIN BUILD
| | | join-table-id=01 plan-id=02 cohort-id=02
| | | build expressions: d_date_sk
-| | | runtime filters: RF018[bloom] <- d_date_sk
+| | | runtime filters: RF018[bloom] <- d_date_sk, RF019[min_max] <- d_date_sk
| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | |
| | 51:EXCHANGE [BROADCAST]
@@ -959,6 +961,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | JOIN BUILD
| | | join-table-id=02 plan-id=03 cohort-id=02
| | | build expressions: c_customer_sk
+| | | runtime filters: RF021[min_max] <- c_customer_sk
| | | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| | |
| | 50:EXCHANGE [HASH(c_customer_sk)]
@@ -988,7 +991,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| Per-Instance Resources: mem-estimate=32.00MB mem-reservation=8.00MB thread-reservation=1
| 23:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| HDFS partitions=1/1 files=2 size=45.09MB
-| runtime filters: RF018[bloom] -> ws_sold_date_sk
+| runtime filters: RF019[min_max] -> ws_sold_date_sk, RF021[min_max] -> ws_bill_customer_sk, RF018[bloom] -> ws_sold_date_sk
| stored statistics:
| table: rows=719.38K size=45.09MB
| columns: all
@@ -1010,7 +1013,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| JOIN BUILD
| | join-table-id=03 plan-id=04 cohort-id=01
| | build expressions: customer_id
-| | runtime filters: RF002[bloom] <- customer_id
+| | runtime filters: RF002[bloom] <- customer_id, RF003[min_max] <- customer_id
| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| |
| 48:EXCHANGE [BROADCAST]
@@ -1060,7 +1063,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | JOIN BUILD
| | | join-table-id=04 plan-id=05 cohort-id=03
| | | build expressions: d_date_sk
-| | | runtime filters: RF014[bloom] <- d_date_sk
+| | | runtime filters: RF014[bloom] <- d_date_sk, RF015[min_max] <- d_date_sk
| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | |
| | 45:EXCHANGE [BROADCAST]
@@ -1096,7 +1099,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | JOIN BUILD
| | | join-table-id=05 plan-id=06 cohort-id=03
| | | build expressions: c_customer_sk
-| | | runtime filters: RF016[bloom] <- c_customer_sk
+| | | runtime filters: RF016[bloom] <- c_customer_sk, RF017[min_max] <- c_customer_sk
| | | mem-estimate=8.50MB mem-reservation=8.50MB spill-buffer=512.00KB thread-reservation=0
| | |
| | 44:EXCHANGE [HASH(c_customer_sk)]
@@ -1109,7 +1112,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | Per-Instance Resources: mem-estimate=16.00MB mem-reservation=8.00MB thread-reservation=1
| | 15:SCAN HDFS [tpcds_parquet.customer, RANDOM]
| | HDFS partitions=1/1 files=1 size=5.49MB
-| | runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id
+| | runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id
| | stored statistics:
| | table: rows=100.00K size=5.49MB
| | columns: all
@@ -1128,7 +1131,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| Per-Instance Resources: mem-estimate=32.00MB mem-reservation=8.00MB thread-reservation=1
| 16:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| HDFS partitions=1/1 files=2 size=45.09MB
-| runtime filters: RF014[bloom] -> ws_sold_date_sk, RF016[bloom] -> ws_bill_customer_sk
+| runtime filters: RF015[min_max] -> ws_sold_date_sk, RF017[min_max] -> ws_bill_customer_sk, RF014[bloom] -> ws_sold_date_sk, RF016[bloom] -> ws_bill_customer_sk
| stored statistics:
| table: rows=719.38K size=45.09MB
| columns: all
@@ -1150,7 +1153,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| JOIN BUILD
| | join-table-id=06 plan-id=07 cohort-id=01
| | build expressions: customer_id
-| | runtime filters: RF004[bloom] <- customer_id
+| | runtime filters: RF004[bloom] <- customer_id, RF005[min_max] <- customer_id
| | mem-estimate=9.50MB mem-reservation=9.50MB spill-buffer=256.00KB thread-reservation=0
| |
| 42:EXCHANGE [BROADCAST]
@@ -1200,7 +1203,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | JOIN BUILD
| | | join-table-id=07 plan-id=08 cohort-id=04
| | | build expressions: c_customer_sk
-| | | runtime filters: RF010[bloom] <- c_customer_sk
+| | | runtime filters: RF010[bloom] <- c_customer_sk, RF011[min_max] <- c_customer_sk
| | | mem-estimate=4.75MB mem-reservation=4.75MB spill-buffer=256.00KB thread-reservation=0
| | |
| | 39:EXCHANGE [HASH(c_customer_sk)]
@@ -1213,7 +1216,7 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | Per-Instance Resources: mem-estimate=16.00MB mem-reservation=8.00MB thread-reservation=1
| | 01:SCAN HDFS [tpcds_parquet.customer, RANDOM]
| | HDFS partitions=1/1 files=1 size=5.49MB
-| | runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id
+| | runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id
| | stored statistics:
| | table: rows=100.00K size=5.49MB
| | columns: all
@@ -1267,10 +1270,10 @@ Per-Instance Resources: mem-estimate=45.02MB mem-reservation=34.00MB thread-rese
| | in pipelines: 03(GETNEXT)
| |
| 02:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
-| runtime filters: RF012[bloom] -> ss_sold_date_sk, RF010[bloom] -> ss_customer_sk
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
+| runtime filters: RF011[min_max] -> ss_customer_sk, RF012[bloom] -> ss_sold_date_sk, RF010[bloom] -> ss_customer_sk
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1317,7 +1320,7 @@ Per-Instance Resources: mem-estimate=38.68MB mem-reservation=34.00MB thread-rese
| JOIN BUILD
| | join-table-id=09 plan-id=10 cohort-id=01
| | build expressions: c_customer_sk
-| | runtime filters: RF006[bloom] <- c_customer_sk
+| | runtime filters: RF006[bloom] <- c_customer_sk, RF007[min_max] <- c_customer_sk
| | mem-estimate=4.75MB mem-reservation=4.75MB spill-buffer=256.00KB thread-reservation=0
| |
| 34:EXCHANGE [HASH(c_customer_sk)]
@@ -1330,7 +1333,7 @@ Per-Instance Resources: mem-estimate=38.68MB mem-reservation=34.00MB thread-rese
| Per-Instance Resources: mem-estimate=16.00MB mem-reservation=8.00MB thread-reservation=1
| 08:SCAN HDFS [tpcds_parquet.customer, RANDOM]
| HDFS partitions=1/1 files=1 size=5.49MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id
+| runtime filters: RF001[min_max] -> tpcds_parquet.customer.c_customer_id, RF003[min_max] -> tpcds_parquet.customer.c_customer_id, RF005[min_max] -> tpcds_parquet.customer.c_customer_id, RF000[bloom] -> tpcds_parquet.customer.c_customer_id, RF002[bloom] -> tpcds_parquet.customer.c_customer_id, RF004[bloom] -> tpcds_parquet.customer.c_customer_id
| stored statistics:
| table: rows=100.00K size=5.49MB
| columns: all
@@ -1384,10 +1387,10 @@ Per-Instance Resources: mem-estimate=16.00MB mem-reservation=2.00MB thread-reser
| in pipelines: 10(GETNEXT)
|
09:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF008[bloom] -> ss_sold_date_sk, RF006[bloom] -> ss_customer_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF007[min_max] -> ss_customer_sk, RF008[bloom] -> ss_sold_date_sk, RF006[bloom] -> ss_customer_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q12.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q12.test
index 15e0b18..63546df 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q12.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q12.test
@@ -64,7 +64,7 @@ PLAN-ROOT SINK
04:HASH JOIN [INNER JOIN]
| hash predicates: ws_sold_date_sk = d_date_sk
| fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| runtime filters: RF000[bloom] <- d_date_sk
+| runtime filters: RF000[bloom] <- d_date_sk, RF001[min_max] <- d_date_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=0,1,2 row-size=232B cardinality=215.82K
| in pipelines: 00(GETNEXT), 02(OPEN)
@@ -85,7 +85,7 @@ PLAN-ROOT SINK
03:HASH JOIN [INNER JOIN]
| hash predicates: ws_item_sk = i_item_sk
| fk/pk conjuncts: ws_item_sk = i_item_sk
-| runtime filters: RF002[bloom] <- i_item_sk
+| runtime filters: RF002[bloom] <- i_item_sk, RF003[min_max] <- i_item_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=0,1 row-size=206B cardinality=215.82K
| in pipelines: 00(GETNEXT), 01(OPEN)
@@ -105,7 +105,7 @@ PLAN-ROOT SINK
|
00:SCAN HDFS [tpcds_parquet.web_sales]
HDFS partitions=1/1 files=2 size=45.09MB
- runtime filters: RF000[bloom] -> ws_sold_date_sk, RF002[bloom] -> ws_item_sk
+ runtime filters: RF001[min_max] -> ws_sold_date_sk, RF003[min_max] -> ws_item_sk, RF000[bloom] -> ws_sold_date_sk, RF002[bloom] -> ws_item_sk
stored statistics:
table: rows=719.38K size=45.09MB
columns: all
@@ -174,7 +174,7 @@ Per-Host Resources: mem-estimate=137.25MB mem-reservation=47.88MB thread-reserva
04:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: ws_sold_date_sk = d_date_sk
| fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| runtime filters: RF000[bloom] <- d_date_sk
+| runtime filters: RF000[bloom] <- d_date_sk, RF001[min_max] <- d_date_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=0,1,2 row-size=232B cardinality=215.82K
| in pipelines: 00(GETNEXT), 02(OPEN)
@@ -202,7 +202,7 @@ Per-Host Resources: mem-estimate=137.25MB mem-reservation=47.88MB thread-reserva
03:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: ws_item_sk = i_item_sk
| fk/pk conjuncts: ws_item_sk = i_item_sk
-| runtime filters: RF002[bloom] <- i_item_sk
+| runtime filters: RF002[bloom] <- i_item_sk, RF003[min_max] <- i_item_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=0,1 row-size=206B cardinality=215.82K
| in pipelines: 00(GETNEXT), 01(OPEN)
@@ -229,7 +229,7 @@ Per-Host Resources: mem-estimate=137.25MB mem-reservation=47.88MB thread-reserva
|
00:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
HDFS partitions=1/1 files=2 size=45.09MB
- runtime filters: RF000[bloom] -> ws_sold_date_sk, RF002[bloom] -> ws_item_sk
+ runtime filters: RF001[min_max] -> ws_sold_date_sk, RF003[min_max] -> ws_item_sk, RF000[bloom] -> ws_sold_date_sk, RF002[bloom] -> ws_item_sk
stored statistics:
table: rows=719.38K size=45.09MB
columns: all
@@ -309,7 +309,7 @@ Per-Instance Resources: mem-estimate=66.00MB mem-reservation=42.00MB thread-rese
| JOIN BUILD
| | join-table-id=00 plan-id=01 cohort-id=01
| | build expressions: d_date_sk
-| | runtime filters: RF000[bloom] <- d_date_sk
+| | runtime filters: RF000[bloom] <- d_date_sk, RF001[min_max] <- d_date_sk
| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| |
| 10:EXCHANGE [BROADCAST]
@@ -345,7 +345,7 @@ Per-Instance Resources: mem-estimate=66.00MB mem-reservation=42.00MB thread-rese
| JOIN BUILD
| | join-table-id=01 plan-id=02 cohort-id=01
| | build expressions: i_item_sk
-| | runtime filters: RF002[bloom] <- i_item_sk
+| | runtime filters: RF002[bloom] <- i_item_sk, RF003[min_max] <- i_item_sk
| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| |
| 09:EXCHANGE [BROADCAST]
@@ -370,7 +370,7 @@ Per-Instance Resources: mem-estimate=66.00MB mem-reservation=42.00MB thread-rese
|
00:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
HDFS partitions=1/1 files=2 size=45.09MB
- runtime filters: RF000[bloom] -> ws_sold_date_sk, RF002[bloom] -> ws_item_sk
+ runtime filters: RF001[min_max] -> ws_sold_date_sk, RF003[min_max] -> ws_item_sk, RF000[bloom] -> ws_sold_date_sk, RF002[bloom] -> ws_item_sk
stored statistics:
table: rows=719.38K size=45.09MB
columns: all
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q13.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q13.test
index c8ea54e..55d6a19 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q13.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q13.test
@@ -65,7 +65,7 @@ PLAN-ROOT SINK
10:HASH JOIN [INNER JOIN]
| hash predicates: ss_store_sk = s_store_sk
| fk/pk conjuncts: ss_store_sk = s_store_sk
-| runtime filters: RF000[bloom] <- s_store_sk
+| runtime filters: RF000[bloom] <- s_store_sk, RF001[min_max] <- s_store_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,0,4,5,3,1 row-size=142B cardinality=2.55K
| in pipelines: 02(GETNEXT), 01(OPEN)
@@ -84,7 +84,7 @@ PLAN-ROOT SINK
| hash predicates: cd_demo_sk = ss_cdemo_sk
| fk/pk conjuncts: cd_demo_sk = ss_cdemo_sk
| other predicates: cd_education_status = 'Advanced Degree' OR cd_marital_status = 'S' AND cd_education_status = 'College' OR hd_dep_count = CAST(1 AS INT), cd_marital_status = 'M' OR cd_marital_status = 'S' AND cd_education_status = 'College' OR hd_dep_count = CAST(1 AS INT), hd_dep_count = CAST(3 AS INT) OR cd_education_status = 'College' OR hd_dep_count = CAST(1 AS INT), cd_education_status = 'Advanced Degree' OR hd_dep_count = CAST(1 AS INT) OR hd_dep_count = CAST(1 AS INT), hd_dep_ [...]
-| runtime filters: RF002[bloom] <- ss_cdemo_sk
+| runtime filters: RF002[bloom] <- ss_cdemo_sk, RF003[min_max] <- ss_cdemo_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=2,0,4,5,3 row-size=138B cardinality=2.55K
| in pipelines: 02(GETNEXT), 00(OPEN)
@@ -93,7 +93,7 @@ PLAN-ROOT SINK
| | hash predicates: ss_hdemo_sk = hd_demo_sk
| | fk/pk conjuncts: ss_hdemo_sk = hd_demo_sk
| | other predicates: hd_dep_count IN (CAST(3 AS INT), CAST(1 AS INT)) OR ss_sales_price <= CAST(200.00 AS DECIMAL(5,2)), hd_dep_count IN (CAST(3 AS INT), CAST(1 AS INT)) OR ss_sales_price >= CAST(150.00 AS DECIMAL(5,2)), hd_dep_count = CAST(3 AS INT) OR ss_sales_price <= CAST(100.00 AS DECIMAL(5,2)) OR hd_dep_count = CAST(1 AS INT), hd_dep_count = CAST(3 AS INT) OR ss_sales_price <= CAST(100.00 AS DECIMAL(5,2)) OR ss_sales_price <= CAST(200.00 AS DECIMAL(5,2)), hd_dep_count = CAST(3 A [...]
-| | runtime filters: RF004[bloom] <- hd_demo_sk
+| | runtime filters: RF004[bloom] <- hd_demo_sk, RF005[min_max] <- hd_demo_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=0,4,5,3 row-size=99B cardinality=3.05K
| | in pipelines: 00(GETNEXT), 03(OPEN)
@@ -136,7 +136,7 @@ PLAN-ROOT SINK
| | hash predicates: ss_addr_sk = ca_address_sk
| | fk/pk conjuncts: ss_addr_sk = ca_address_sk
| | other predicates: ca_state IN ('TX', 'OH', 'TX') OR ss_net_profit <= CAST(300 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(250 AS DECIMAL(5,0)), ca_state IN ('TX', 'OH', 'TX') OR ss_net_profit <= CAST(300 AS DECIMAL(5,0)) OR ss_net_profit >= CAST(50 AS DECIMAL(3,0)), ca_state IN ('TX', 'OH', 'TX') OR ss_net_profit >= CAST(150 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(250 AS DECIMAL(5,0)), ca_state IN ('TX', 'OH', 'TX') OR ss_net_profit >= CAST(150 AS DECIMAL(5,0)) OR ss_net_profit >= [...]
-| | runtime filters: RF008[bloom] <- ca_address_sk
+| | runtime filters: RF008[bloom] <- ca_address_sk, RF009[min_max] <- ca_address_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=0,4 row-size=83B cardinality=50.98K
| | in pipelines: 00(GETNEXT), 04(OPEN)
@@ -155,11 +155,11 @@ PLAN-ROOT SINK
| | in pipelines: 04(GETNEXT)
| |
| 00:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| predicates: ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(300 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(250 AS DECIMAL(5,0)), ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(300 AS DECIMAL(5,0)) OR ss_net_profit >= CAST(50 AS DECIMAL(3,0)), ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit >= CAST(150 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(250 AS DECIMAL(5,0)), ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit >= CAST [...]
-| runtime filters: RF000[bloom] -> ss_store_sk, RF004[bloom] -> ss_hdemo_sk, RF006[bloom] -> ss_sold_date_sk, RF008[bloom] -> ss_addr_sk
+| runtime filters: RF001[min_max] -> ss_store_sk, RF005[min_max] -> ss_hdemo_sk, RF009[min_max] -> ss_addr_sk, RF000[bloom] -> ss_store_sk, RF004[bloom] -> ss_hdemo_sk, RF006[bloom] -> ss_sold_date_sk, RF008[bloom] -> ss_addr_sk
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -171,7 +171,7 @@ PLAN-ROOT SINK
02:SCAN HDFS [tpcds_parquet.customer_demographics]
HDFS partitions=1/1 files=1 size=7.49MB
predicates: cd_marital_status = 'M' OR cd_marital_status = 'S' AND cd_education_status = 'College' OR cd_marital_status = 'W' AND cd_education_status = '2 yr Degree', cd_education_status = 'Advanced Degree' OR cd_marital_status = 'S' AND cd_education_status = 'College' OR cd_marital_status = 'W' AND cd_education_status = '2 yr Degree'
- runtime filters: RF002[bloom] -> cd_demo_sk
+ runtime filters: RF003[min_max] -> cd_demo_sk, RF002[bloom] -> cd_demo_sk
stored statistics:
table: rows=1.92M size=7.49MB
columns: all
@@ -179,4 +179,417 @@ PLAN-ROOT SINK
mem-estimate=48.00MB mem-reservation=8.00MB thread-reservation=1
tuple-ids=2 row-size=39B cardinality=181.75K
in pipelines: 02(GETNEXT)
+---- DISTRIBUTEDPLAN
+Max Per-Host Resource Reservation: Memory=32.33MB Threads=14
+Per-Host Resource Estimates: Memory=344MB
+F07:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
+| Per-Host Resources: mem-estimate=10.02MB mem-reservation=0B thread-reservation=1
+PLAN-ROOT SINK
+| output exprs: avg(ss_quantity), avg(ss_ext_sales_price), avg(ss_ext_wholesale_cost), sum(ss_ext_wholesale_cost)
+| mem-estimate=0B mem-reservation=0B thread-reservation=0
+|
+19:AGGREGATE [FINALIZE]
+| output: avg:merge(ss_quantity), avg:merge(ss_ext_sales_price), avg:merge(ss_ext_wholesale_cost), sum:merge(ss_ext_wholesale_cost)
+| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=7 row-size=40B cardinality=1
+| in pipelines: 19(GETNEXT), 11(OPEN)
+|
+18:EXCHANGE [UNPARTITIONED]
+| mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| tuple-ids=6 row-size=40B cardinality=1
+| in pipelines: 11(GETNEXT)
+|
+F05:PLAN FRAGMENT [HASH(ss_cdemo_sk)] hosts=3 instances=3
+Per-Host Resources: mem-estimate=19.12MB mem-reservation=8.69MB thread-reservation=1 runtime-filters-memory=2.00MB
+11:AGGREGATE
+| output: avg(CAST(ss_quantity AS BIGINT)), avg(ss_ext_sales_price), avg(ss_ext_wholesale_cost), sum(ss_ext_wholesale_cost)
+| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=6 row-size=40B cardinality=1
+| in pipelines: 11(GETNEXT), 00(OPEN)
+|
+10:HASH JOIN [INNER JOIN, BROADCAST]
+| hash predicates: ss_store_sk = s_store_sk
+| fk/pk conjuncts: ss_store_sk = s_store_sk
+| runtime filters: RF000[bloom] <- s_store_sk, RF001[min_max] <- s_store_sk
+| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,4,5,3,2,1 row-size=142B cardinality=2.55K
+| in pipelines: 00(GETNEXT), 01(OPEN)
+|
+|--17:EXCHANGE [BROADCAST]
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=1 row-size=4B cardinality=12
+| | in pipelines: 01(GETNEXT)
+| |
+| F06:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Host Resources: mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=2
+| 01:SCAN HDFS [tpcds_parquet.store, RANDOM]
+| HDFS partitions=1/1 files=1 size=9.93KB
+| stored statistics:
+| table: rows=12 size=9.93KB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=12
+| mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=1
+| tuple-ids=1 row-size=4B cardinality=12
+| in pipelines: 01(GETNEXT)
+|
+09:HASH JOIN [INNER JOIN, PARTITIONED]
+| hash predicates: ss_cdemo_sk = cd_demo_sk
+| fk/pk conjuncts: ss_cdemo_sk = cd_demo_sk
+| other predicates: cd_education_status = 'Advanced Degree' OR cd_marital_status = 'S' AND cd_education_status = 'College' OR hd_dep_count = CAST(1 AS INT), cd_marital_status = 'M' OR cd_marital_status = 'S' AND cd_education_status = 'College' OR hd_dep_count = CAST(1 AS INT), hd_dep_count = CAST(3 AS INT) OR cd_education_status = 'College' OR hd_dep_count = CAST(1 AS INT), cd_education_status = 'Advanced Degree' OR hd_dep_count = CAST(1 AS INT) OR hd_dep_count = CAST(1 AS INT), hd_dep_ [...]
+| runtime filters: RF002[bloom] <- cd_demo_sk, RF003[min_max] <- cd_demo_sk
+| mem-estimate=4.75MB mem-reservation=4.75MB spill-buffer=256.00KB thread-reservation=0
+| tuple-ids=0,4,5,3,2 row-size=138B cardinality=2.55K
+| in pipelines: 00(GETNEXT), 02(OPEN)
+|
+|--16:EXCHANGE [HASH(cd_demo_sk)]
+| | mem-estimate=6.73MB mem-reservation=0B thread-reservation=0
+| | tuple-ids=2 row-size=39B cardinality=181.75K
+| | in pipelines: 02(GETNEXT)
+| |
+| F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Host Resources: mem-estimate=48.00MB mem-reservation=8.00MB thread-reservation=2
+| 02:SCAN HDFS [tpcds_parquet.customer_demographics, RANDOM]
+| HDFS partitions=1/1 files=1 size=7.49MB
+| predicates: cd_marital_status = 'M' OR cd_marital_status = 'S' AND cd_education_status = 'College' OR cd_marital_status = 'W' AND cd_education_status = '2 yr Degree', cd_education_status = 'Advanced Degree' OR cd_marital_status = 'S' AND cd_education_status = 'College' OR cd_marital_status = 'W' AND cd_education_status = '2 yr Degree'
+| stored statistics:
+| table: rows=1.92M size=7.49MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=1.92M
+| mem-estimate=48.00MB mem-reservation=8.00MB thread-reservation=1
+| tuple-ids=2 row-size=39B cardinality=181.75K
+| in pipelines: 02(GETNEXT)
+|
+15:EXCHANGE [HASH(ss_cdemo_sk)]
+| mem-estimate=443.39KB mem-reservation=0B thread-reservation=0
+| tuple-ids=0,4,5,3 row-size=99B cardinality=3.05K
+| in pipelines: 00(GETNEXT)
+|
+F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
+Per-Host Resources: mem-estimate=139.26MB mem-reservation=14.81MB thread-reservation=2 runtime-filters-memory=5.00MB
+08:HASH JOIN [INNER JOIN, BROADCAST]
+| hash predicates: ss_hdemo_sk = hd_demo_sk
+| fk/pk conjuncts: ss_hdemo_sk = hd_demo_sk
+| other predicates: hd_dep_count IN (CAST(3 AS INT), CAST(1 AS INT)) OR ss_sales_price <= CAST(200.00 AS DECIMAL(5,2)), hd_dep_count IN (CAST(3 AS INT), CAST(1 AS INT)) OR ss_sales_price >= CAST(150.00 AS DECIMAL(5,2)), hd_dep_count = CAST(3 AS INT) OR ss_sales_price <= CAST(100.00 AS DECIMAL(5,2)) OR hd_dep_count = CAST(1 AS INT), hd_dep_count = CAST(3 AS INT) OR ss_sales_price <= CAST(100.00 AS DECIMAL(5,2)) OR ss_sales_price <= CAST(200.00 AS DECIMAL(5,2)), hd_dep_count = CAST(3 AS I [...]
+| runtime filters: RF004[bloom] <- hd_demo_sk, RF005[min_max] <- hd_demo_sk
+| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,4,5,3 row-size=99B cardinality=3.05K
+| in pipelines: 00(GETNEXT), 03(OPEN)
+|
+|--14:EXCHANGE [BROADCAST]
+| | mem-estimate=28.88KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=3 row-size=8B cardinality=2.16K
+| | in pipelines: 03(GETNEXT)
+| |
+| F03:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Host Resources: mem-estimate=32.00MB mem-reservation=64.00KB thread-reservation=2
+| 03:SCAN HDFS [tpcds_parquet.household_demographics, RANDOM]
+| HDFS partitions=1/1 files=1 size=41.69KB
+| predicates: hd_dep_count IN (CAST(3 AS INT), CAST(1 AS INT), CAST(1 AS INT))
+| stored statistics:
+| table: rows=7.20K size=41.69KB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=7.20K
+| parquet statistics predicates: hd_dep_count IN (CAST(3 AS INT), CAST(1 AS INT), CAST(1 AS INT))
+| parquet dictionary predicates: hd_dep_count IN (CAST(3 AS INT), CAST(1 AS INT), CAST(1 AS INT))
+| mem-estimate=32.00MB mem-reservation=64.00KB thread-reservation=1
+| tuple-ids=3 row-size=8B cardinality=2.16K
+| in pipelines: 03(GETNEXT)
+|
+07:HASH JOIN [INNER JOIN, BROADCAST]
+| hash predicates: ss_sold_date_sk = d_date_sk
+| fk/pk conjuncts: ss_sold_date_sk = d_date_sk
+| runtime filters: RF006[bloom] <- d_date_sk
+| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,4,5 row-size=91B cardinality=10.43K
+| in pipelines: 00(GETNEXT), 05(OPEN)
+|
+|--13:EXCHANGE [BROADCAST]
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=5 row-size=8B cardinality=373
+| | in pipelines: 05(GETNEXT)
+| |
+| F02:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Host Resources: mem-estimate=32.00MB mem-reservation=512.00KB thread-reservation=2
+| 05:SCAN HDFS [tpcds_parquet.date_dim, RANDOM]
+| HDFS partitions=1/1 files=1 size=2.15MB
+| predicates: d_year = CAST(2001 AS INT)
+| stored statistics:
+| table: rows=73.05K size=2.15MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=73.05K
+| parquet statistics predicates: d_year = CAST(2001 AS INT)
+| parquet dictionary predicates: d_year = CAST(2001 AS INT)
+| mem-estimate=32.00MB mem-reservation=512.00KB thread-reservation=1
+| tuple-ids=5 row-size=8B cardinality=373
+| in pipelines: 05(GETNEXT)
+|
+06:HASH JOIN [INNER JOIN, BROADCAST]
+| hash predicates: ss_addr_sk = ca_address_sk
+| fk/pk conjuncts: ss_addr_sk = ca_address_sk
+| other predicates: ca_state IN ('TX', 'OH', 'TX') OR ss_net_profit <= CAST(300 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(250 AS DECIMAL(5,0)), ca_state IN ('TX', 'OH', 'TX') OR ss_net_profit <= CAST(300 AS DECIMAL(5,0)) OR ss_net_profit >= CAST(50 AS DECIMAL(3,0)), ca_state IN ('TX', 'OH', 'TX') OR ss_net_profit >= CAST(150 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(250 AS DECIMAL(5,0)), ca_state IN ('TX', 'OH', 'TX') OR ss_net_profit >= CAST(150 AS DECIMAL(5,0)) OR ss_net_profit >= CAS [...]
+| runtime filters: RF008[bloom] <- ca_address_sk, RF009[min_max] <- ca_address_sk
+| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,4 row-size=83B cardinality=50.98K
+| in pipelines: 00(GETNEXT), 04(OPEN)
+|
+|--12:EXCHANGE [BROADCAST]
+| | mem-estimate=417.54KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=4 row-size=43B cardinality=8.82K
+| | in pipelines: 04(GETNEXT)
+| |
+| F01:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Host Resources: mem-estimate=48.00MB mem-reservation=256.00KB thread-reservation=2
+| 04:SCAN HDFS [tpcds_parquet.customer_address, RANDOM]
+| HDFS partitions=1/1 files=1 size=1.16MB
+| predicates: ca_state IN ('TX', 'OH', 'TX', 'OR', 'NM', 'KY', 'VA', 'TX', 'MS'), ca_country = 'United States'
+| stored statistics:
+| table: rows=50.00K size=1.16MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=50.00K
+| parquet statistics predicates: ca_state IN ('TX', 'OH', 'TX', 'OR', 'NM', 'KY', 'VA', 'TX', 'MS'), ca_country = 'United States'
+| parquet dictionary predicates: ca_state IN ('TX', 'OH', 'TX', 'OR', 'NM', 'KY', 'VA', 'TX', 'MS'), ca_country = 'United States'
+| mem-estimate=48.00MB mem-reservation=256.00KB thread-reservation=1
+| tuple-ids=4 row-size=43B cardinality=8.82K
+| in pipelines: 04(GETNEXT)
+|
+00:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ predicates: ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(300 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(250 AS DECIMAL(5,0)), ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(300 AS DECIMAL(5,0)) OR ss_net_profit >= CAST(50 AS DECIMAL(3,0)), ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit >= CAST(150 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(250 AS DECIMAL(5,0)), ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit >= CAST(15 [...]
+ runtime filters: RF001[min_max] -> ss_store_sk, RF003[min_max] -> ss_cdemo_sk, RF005[min_max] -> ss_hdemo_sk, RF009[min_max] -> ss_addr_sk, RF000[bloom] -> ss_store_sk, RF002[bloom] -> ss_cdemo_sk, RF004[bloom] -> ss_hdemo_sk, RF006[bloom] -> ss_sold_date_sk, RF008[bloom] -> ss_addr_sk
+ stored statistics:
+ table: rows=2.88M size=200.96MB
+ partitions: 1824/1824 rows=2.88M
+ columns: all
+ extrapolated-rows=disabled max-scan-range-rows=130.09K
+ parquet dictionary predicates: ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(300 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(250 AS DECIMAL(5,0)), ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(300 AS DECIMAL(5,0)) OR ss_net_profit >= CAST(50 AS DECIMAL(3,0)), ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit >= CAST(150 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(250 AS DECIMAL(5,0)), ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_ne [...]
+ mem-estimate=128.00MB mem-reservation=4.00MB thread-reservation=1
+ tuple-ids=0 row-size=40B cardinality=288.04K
+ in pipelines: 00(GETNEXT)
+---- PARALLELPLANS
+Max Per-Host Resource Reservation: Memory=49.08MB Threads=16
+Per-Host Resource Estimates: Memory=190MB
+F07:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
+| Per-Instance Resources: mem-estimate=10.02MB mem-reservation=0B thread-reservation=1
+PLAN-ROOT SINK
+| output exprs: avg(ss_quantity), avg(ss_ext_sales_price), avg(ss_ext_wholesale_cost), sum(ss_ext_wholesale_cost)
+| mem-estimate=0B mem-reservation=0B thread-reservation=0
+|
+19:AGGREGATE [FINALIZE]
+| output: avg:merge(ss_quantity), avg:merge(ss_ext_sales_price), avg:merge(ss_ext_wholesale_cost), sum:merge(ss_ext_wholesale_cost)
+| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=7 row-size=40B cardinality=1
+| in pipelines: 19(GETNEXT), 11(OPEN)
+|
+18:EXCHANGE [UNPARTITIONED]
+| mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| tuple-ids=6 row-size=40B cardinality=1
+| in pipelines: 11(GETNEXT)
+|
+F05:PLAN FRAGMENT [HASH(ss_cdemo_sk)] hosts=3 instances=6
+Per-Instance Resources: mem-estimate=10.77MB mem-reservation=0B thread-reservation=1
+11:AGGREGATE
+| output: avg(CAST(ss_quantity AS BIGINT)), avg(ss_ext_sales_price), avg(ss_ext_wholesale_cost), sum(ss_ext_wholesale_cost)
+| mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
+| tuple-ids=6 row-size=40B cardinality=1
+| in pipelines: 11(GETNEXT), 00(OPEN)
+|
+10:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=00
+| hash predicates: ss_store_sk = s_store_sk
+| fk/pk conjuncts: ss_store_sk = s_store_sk
+| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,4,5,3,2,1 row-size=142B cardinality=2.55K
+| in pipelines: 00(GETNEXT), 01(OPEN)
+|
+|--F08:PLAN FRAGMENT [HASH(ss_cdemo_sk)] hosts=3 instances=3
+| | Per-Instance Resources: mem-estimate=4.89MB mem-reservation=4.88MB thread-reservation=1 runtime-filters-memory=1.00MB
+| JOIN BUILD
+| | join-table-id=00 plan-id=01 cohort-id=01
+| | build expressions: s_store_sk
+| | runtime filters: RF000[bloom] <- s_store_sk, RF001[min_max] <- s_store_sk
+| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
+| |
+| 17:EXCHANGE [BROADCAST]
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=1 row-size=4B cardinality=12
+| | in pipelines: 01(GETNEXT)
+| |
+| F06:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Instance Resources: mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=1
+| 01:SCAN HDFS [tpcds_parquet.store, RANDOM]
+| HDFS partitions=1/1 files=1 size=9.93KB
+| stored statistics:
+| table: rows=12 size=9.93KB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=12
+| mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=0
+| tuple-ids=1 row-size=4B cardinality=12
+| in pipelines: 01(GETNEXT)
+|
+09:HASH JOIN [INNER JOIN, PARTITIONED]
+| hash-table-id=01
+| hash predicates: ss_cdemo_sk = cd_demo_sk
+| fk/pk conjuncts: ss_cdemo_sk = cd_demo_sk
+| other predicates: cd_education_status = 'Advanced Degree' OR cd_marital_status = 'S' AND cd_education_status = 'College' OR hd_dep_count = CAST(1 AS INT), cd_marital_status = 'M' OR cd_marital_status = 'S' AND cd_education_status = 'College' OR hd_dep_count = CAST(1 AS INT), hd_dep_count = CAST(3 AS INT) OR cd_education_status = 'College' OR hd_dep_count = CAST(1 AS INT), cd_education_status = 'Advanced Degree' OR hd_dep_count = CAST(1 AS INT) OR hd_dep_count = CAST(1 AS INT), hd_dep_ [...]
+| mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0
+| tuple-ids=0,4,5,3,2 row-size=138B cardinality=2.55K
+| in pipelines: 00(GETNEXT), 02(OPEN)
+|
+|--F09:PLAN FRAGMENT [HASH(ss_cdemo_sk)] hosts=3 instances=6
+| | Per-Instance Resources: mem-estimate=10.60MB mem-reservation=3.88MB thread-reservation=1 runtime-filters-memory=1.00MB
+| JOIN BUILD
+| | join-table-id=01 plan-id=02 cohort-id=01
+| | build expressions: cd_demo_sk
+| | runtime filters: RF002[bloom] <- cd_demo_sk, RF003[min_max] <- cd_demo_sk
+| | mem-estimate=2.88MB mem-reservation=2.88MB spill-buffer=128.00KB thread-reservation=0
+| |
+| 16:EXCHANGE [HASH(cd_demo_sk)]
+| | mem-estimate=6.73MB mem-reservation=0B thread-reservation=0
+| | tuple-ids=2 row-size=39B cardinality=181.75K
+| | in pipelines: 02(GETNEXT)
+| |
+| F04:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Instance Resources: mem-estimate=16.00MB mem-reservation=8.00MB thread-reservation=1
+| 02:SCAN HDFS [tpcds_parquet.customer_demographics, RANDOM]
+| HDFS partitions=1/1 files=1 size=7.49MB
+| predicates: cd_marital_status = 'M' OR cd_marital_status = 'S' AND cd_education_status = 'College' OR cd_marital_status = 'W' AND cd_education_status = '2 yr Degree', cd_education_status = 'Advanced Degree' OR cd_marital_status = 'S' AND cd_education_status = 'College' OR cd_marital_status = 'W' AND cd_education_status = '2 yr Degree'
+| stored statistics:
+| table: rows=1.92M size=7.49MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=1.92M
+| mem-estimate=16.00MB mem-reservation=8.00MB thread-reservation=0
+| tuple-ids=2 row-size=39B cardinality=181.75K
+| in pipelines: 02(GETNEXT)
+|
+15:EXCHANGE [HASH(ss_cdemo_sk)]
+| mem-estimate=788.39KB mem-reservation=0B thread-reservation=0
+| tuple-ids=0,4,5,3 row-size=99B cardinality=3.05K
+| in pipelines: 00(GETNEXT)
+|
+F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=6
+Per-Host Shared Resources: mem-estimate=5.00MB mem-reservation=5.00MB thread-reservation=0 runtime-filters-memory=5.00MB
+Per-Instance Resources: mem-estimate=16.00MB mem-reservation=4.00MB thread-reservation=1
+08:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=02
+| hash predicates: ss_hdemo_sk = hd_demo_sk
+| fk/pk conjuncts: ss_hdemo_sk = hd_demo_sk
+| other predicates: hd_dep_count IN (CAST(3 AS INT), CAST(1 AS INT)) OR ss_sales_price <= CAST(200.00 AS DECIMAL(5,2)), hd_dep_count IN (CAST(3 AS INT), CAST(1 AS INT)) OR ss_sales_price >= CAST(150.00 AS DECIMAL(5,2)), hd_dep_count = CAST(3 AS INT) OR ss_sales_price <= CAST(100.00 AS DECIMAL(5,2)) OR hd_dep_count = CAST(1 AS INT), hd_dep_count = CAST(3 AS INT) OR ss_sales_price <= CAST(100.00 AS DECIMAL(5,2)) OR ss_sales_price <= CAST(200.00 AS DECIMAL(5,2)), hd_dep_count = CAST(3 AS I [...]
+| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,4,5,3 row-size=99B cardinality=3.05K
+| in pipelines: 00(GETNEXT), 03(OPEN)
+|
+|--F10:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
+| | Per-Instance Resources: mem-estimate=4.90MB mem-reservation=4.88MB thread-reservation=1 runtime-filters-memory=1.00MB
+| JOIN BUILD
+| | join-table-id=02 plan-id=03 cohort-id=01
+| | build expressions: hd_demo_sk
+| | runtime filters: RF004[bloom] <- hd_demo_sk, RF005[min_max] <- hd_demo_sk
+| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
+| |
+| 14:EXCHANGE [BROADCAST]
+| | mem-estimate=28.88KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=3 row-size=8B cardinality=2.16K
+| | in pipelines: 03(GETNEXT)
+| |
+| F03:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Instance Resources: mem-estimate=16.00MB mem-reservation=64.00KB thread-reservation=1
+| 03:SCAN HDFS [tpcds_parquet.household_demographics, RANDOM]
+| HDFS partitions=1/1 files=1 size=41.69KB
+| predicates: hd_dep_count IN (CAST(3 AS INT), CAST(1 AS INT), CAST(1 AS INT))
+| stored statistics:
+| table: rows=7.20K size=41.69KB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=7.20K
+| parquet statistics predicates: hd_dep_count IN (CAST(3 AS INT), CAST(1 AS INT), CAST(1 AS INT))
+| parquet dictionary predicates: hd_dep_count IN (CAST(3 AS INT), CAST(1 AS INT), CAST(1 AS INT))
+| mem-estimate=16.00MB mem-reservation=64.00KB thread-reservation=0
+| tuple-ids=3 row-size=8B cardinality=2.16K
+| in pipelines: 03(GETNEXT)
+|
+07:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=03
+| hash predicates: ss_sold_date_sk = d_date_sk
+| fk/pk conjuncts: ss_sold_date_sk = d_date_sk
+| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,4,5 row-size=91B cardinality=10.43K
+| in pipelines: 00(GETNEXT), 05(OPEN)
+|
+|--F11:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
+| | Per-Instance Resources: mem-estimate=4.89MB mem-reservation=4.88MB thread-reservation=1 runtime-filters-memory=1.00MB
+| JOIN BUILD
+| | join-table-id=03 plan-id=04 cohort-id=01
+| | build expressions: d_date_sk
+| | runtime filters: RF006[bloom] <- d_date_sk
+| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
+| |
+| 13:EXCHANGE [BROADCAST]
+| | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=5 row-size=8B cardinality=373
+| | in pipelines: 05(GETNEXT)
+| |
+| F02:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Instance Resources: mem-estimate=16.00MB mem-reservation=512.00KB thread-reservation=1
+| 05:SCAN HDFS [tpcds_parquet.date_dim, RANDOM]
+| HDFS partitions=1/1 files=1 size=2.15MB
+| predicates: d_year = CAST(2001 AS INT)
+| stored statistics:
+| table: rows=73.05K size=2.15MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=73.05K
+| parquet statistics predicates: d_year = CAST(2001 AS INT)
+| parquet dictionary predicates: d_year = CAST(2001 AS INT)
+| mem-estimate=16.00MB mem-reservation=512.00KB thread-reservation=0
+| tuple-ids=5 row-size=8B cardinality=373
+| in pipelines: 05(GETNEXT)
+|
+06:HASH JOIN [INNER JOIN, BROADCAST]
+| hash-table-id=04
+| hash predicates: ss_addr_sk = ca_address_sk
+| fk/pk conjuncts: ss_addr_sk = ca_address_sk
+| other predicates: ca_state IN ('TX', 'OH', 'TX') OR ss_net_profit <= CAST(300 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(250 AS DECIMAL(5,0)), ca_state IN ('TX', 'OH', 'TX') OR ss_net_profit <= CAST(300 AS DECIMAL(5,0)) OR ss_net_profit >= CAST(50 AS DECIMAL(3,0)), ca_state IN ('TX', 'OH', 'TX') OR ss_net_profit >= CAST(150 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(250 AS DECIMAL(5,0)), ca_state IN ('TX', 'OH', 'TX') OR ss_net_profit >= CAST(150 AS DECIMAL(5,0)) OR ss_net_profit >= CAS [...]
+| mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0
+| tuple-ids=0,4 row-size=83B cardinality=50.98K
+| in pipelines: 00(GETNEXT), 04(OPEN)
+|
+|--F12:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
+| | Per-Instance Resources: mem-estimate=5.28MB mem-reservation=4.88MB thread-reservation=1 runtime-filters-memory=1.00MB
+| JOIN BUILD
+| | join-table-id=04 plan-id=05 cohort-id=01
+| | build expressions: ca_address_sk
+| | runtime filters: RF008[bloom] <- ca_address_sk, RF009[min_max] <- ca_address_sk
+| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
+| |
+| 12:EXCHANGE [BROADCAST]
+| | mem-estimate=417.54KB mem-reservation=0B thread-reservation=0
+| | tuple-ids=4 row-size=43B cardinality=8.82K
+| | in pipelines: 04(GETNEXT)
+| |
+| F01:PLAN FRAGMENT [RANDOM] hosts=1 instances=1
+| Per-Instance Resources: mem-estimate=16.00MB mem-reservation=256.00KB thread-reservation=1
+| 04:SCAN HDFS [tpcds_parquet.customer_address, RANDOM]
+| HDFS partitions=1/1 files=1 size=1.16MB
+| predicates: ca_state IN ('TX', 'OH', 'TX', 'OR', 'NM', 'KY', 'VA', 'TX', 'MS'), ca_country = 'United States'
+| stored statistics:
+| table: rows=50.00K size=1.16MB
+| columns: all
+| extrapolated-rows=disabled max-scan-range-rows=50.00K
+| parquet statistics predicates: ca_state IN ('TX', 'OH', 'TX', 'OR', 'NM', 'KY', 'VA', 'TX', 'MS'), ca_country = 'United States'
+| parquet dictionary predicates: ca_state IN ('TX', 'OH', 'TX', 'OR', 'NM', 'KY', 'VA', 'TX', 'MS'), ca_country = 'United States'
+| mem-estimate=16.00MB mem-reservation=256.00KB thread-reservation=0
+| tuple-ids=4 row-size=43B cardinality=8.82K
+| in pipelines: 04(GETNEXT)
+|
+00:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ predicates: ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(300 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(250 AS DECIMAL(5,0)), ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(300 AS DECIMAL(5,0)) OR ss_net_profit >= CAST(50 AS DECIMAL(3,0)), ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit >= CAST(150 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(250 AS DECIMAL(5,0)), ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit >= CAST(15 [...]
+ runtime filters: RF001[min_max] -> ss_store_sk, RF003[min_max] -> ss_cdemo_sk, RF005[min_max] -> ss_hdemo_sk, RF009[min_max] -> ss_addr_sk, RF000[bloom] -> ss_store_sk, RF002[bloom] -> ss_cdemo_sk, RF004[bloom] -> ss_hdemo_sk, RF006[bloom] -> ss_sold_date_sk, RF008[bloom] -> ss_addr_sk
+ stored statistics:
+ table: rows=2.88M size=200.96MB
+ partitions: 1824/1824 rows=2.88M
+ columns: all
+ extrapolated-rows=disabled max-scan-range-rows=130.09K
+ parquet dictionary predicates: ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(300 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(250 AS DECIMAL(5,0)), ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(300 AS DECIMAL(5,0)) OR ss_net_profit >= CAST(50 AS DECIMAL(3,0)), ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_net_profit >= CAST(150 AS DECIMAL(5,0)) OR ss_net_profit <= CAST(250 AS DECIMAL(5,0)), ss_net_profit <= CAST(200 AS DECIMAL(5,0)) OR ss_ne [...]
+ mem-estimate=16.00MB mem-reservation=4.00MB thread-reservation=0
+ tuple-ids=0 row-size=40B cardinality=288.04K
+ in pipelines: 00(GETNEXT)
====
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q14a.test b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q14a.test
index 4889797..a05b176 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q14a.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/tpcds/tpcds-q14a.test
@@ -167,6 +167,7 @@ PLAN-ROOT SINK
| | |--124:HASH JOIN [INNER JOIN]
| | | | hash predicates: ws_sold_date_sk = d_date_sk
| | | | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
+| | | | runtime filters: RF107[min_max] <- d_date_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=94,95 row-size=20B cardinality=719.38K
| | | | in pipelines: 122(GETNEXT), 123(OPEN)
@@ -186,6 +187,7 @@ PLAN-ROOT SINK
| | | |
| | | 122:SCAN HDFS [tpcds_parquet.web_sales]
| | | HDFS partitions=1/1 files=2 size=45.09MB
+| | | runtime filters: RF107[min_max] -> ws_sold_date_sk
| | | stored statistics:
| | | table: rows=719.38K size=45.09MB
| | | columns: all
@@ -197,6 +199,7 @@ PLAN-ROOT SINK
| | |--121:HASH JOIN [INNER JOIN]
| | | | hash predicates: cs_sold_date_sk = d_date_sk
| | | | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
+| | | | runtime filters: RF105[min_max] <- d_date_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=92,93 row-size=20B cardinality=1.44M
| | | | in pipelines: 119(GETNEXT), 120(OPEN)
@@ -216,6 +219,7 @@ PLAN-ROOT SINK
| | | |
| | | 119:SCAN HDFS [tpcds_parquet.catalog_sales]
| | | HDFS partitions=1/1 files=3 size=96.62MB
+| | | runtime filters: RF105[min_max] -> cs_sold_date_sk
| | | stored statistics:
| | | table: rows=1.44M size=96.62MB
| | | columns: all
@@ -245,9 +249,9 @@ PLAN-ROOT SINK
| | | in pipelines: 117(GETNEXT)
| | |
| | 116:SCAN HDFS [tpcds_parquet.store_sales]
-| | HDFS partitions=1824/1824 files=1824 size=200.95MB
+| | HDFS partitions=1824/1824 files=1824 size=200.96MB
| | stored statistics:
-| | table: rows=2.88M size=200.95MB
+| | table: rows=2.88M size=200.96MB
| | partitions: 1824/1824 rows=2.88M
| | columns: all
| | extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -264,6 +268,7 @@ PLAN-ROOT SINK
| |
| 113:HASH JOIN [LEFT SEMI JOIN]
| | hash predicates: ws_item_sk = tpcds_parquet.item.i_item_sk
+| | runtime filters: RF073[min_max] <- tpcds_parquet.item.i_item_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=68,70,69 row-size=52B cardinality=42.85K
| | in pipelines: 85(GETNEXT), 112(OPEN)
@@ -277,6 +282,7 @@ PLAN-ROOT SINK
| | 109:HASH JOIN [INNER JOIN]
| | | hash predicates: iss.i_brand_id = i_brand_id, iss.i_category_id = i_category_id, iss.i_class_id = i_class_id
| | | fk/pk conjuncts: iss.i_brand_id = i_brand_id, iss.i_category_id = i_category_id, iss.i_class_id = i_class_id
+| | | runtime filters: RF081[min_max] <- i_brand_id, RF082[min_max] <- i_category_id, RF083[min_max] <- i_class_id
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=75,71 row-size=32B cardinality=148.80K
| | | in pipelines: 94(GETNEXT), 88(OPEN)
@@ -306,6 +312,7 @@ PLAN-ROOT SINK
| | | 104:HASH JOIN [INNER JOIN]
| | | | hash predicates: ws_sold_date_sk = d3.d_date_sk
| | | | fk/pk conjuncts: ws_sold_date_sk = d3.d_date_sk
+| | | | runtime filters: RF099[min_max] <- d3.d_date_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=81,82,83 row-size=40B cardinality=719.38K
| | | | in pipelines: 100(GETNEXT), 102(OPEN)
@@ -326,6 +333,7 @@ PLAN-ROOT SINK
| | | 103:HASH JOIN [INNER JOIN]
| | | | hash predicates: ws_item_sk = iws.i_item_sk
| | | | fk/pk conjuncts: ws_item_sk = iws.i_item_sk
+| | | | runtime filters: RF101[min_max] <- iws.i_item_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=81,82 row-size=32B cardinality=719.38K
| | | | in pipelines: 100(GETNEXT), 101(OPEN)
@@ -342,6 +350,7 @@ PLAN-ROOT SINK
| | | |
| | | 100:SCAN HDFS [tpcds_parquet.web_sales]
| | | HDFS partitions=1/1 files=2 size=45.09MB
+| | | runtime filters: RF099[min_max] -> ws_sold_date_sk, RF101[min_max] -> ws_item_sk
| | | stored statistics:
| | | table: rows=719.38K size=45.09MB
| | | columns: all
@@ -365,6 +374,7 @@ PLAN-ROOT SINK
| | | 99:HASH JOIN [INNER JOIN]
| | | | hash predicates: cs_sold_date_sk = d2.d_date_sk
| | | | fk/pk conjuncts: cs_sold_date_sk = d2.d_date_sk
+| | | | runtime filters: RF095[min_max] <- d2.d_date_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=77,78,79 row-size=40B cardinality=1.44M
| | | | in pipelines: 95(GETNEXT), 97(OPEN)
@@ -385,6 +395,7 @@ PLAN-ROOT SINK
| | | 98:HASH JOIN [INNER JOIN]
| | | | hash predicates: cs_item_sk = ics.i_item_sk
| | | | fk/pk conjuncts: cs_item_sk = ics.i_item_sk
+| | | | runtime filters: RF097[min_max] <- ics.i_item_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=77,78 row-size=32B cardinality=1.44M
| | | | in pipelines: 95(GETNEXT), 96(OPEN)
@@ -401,6 +412,7 @@ PLAN-ROOT SINK
| | | |
| | | 95:SCAN HDFS [tpcds_parquet.catalog_sales]
| | | HDFS partitions=1/1 files=3 size=96.62MB
+| | | runtime filters: RF095[min_max] -> cs_sold_date_sk, RF097[min_max] -> cs_item_sk
| | | stored statistics:
| | | table: rows=1.44M size=96.62MB
| | | columns: all
@@ -438,12 +450,14 @@ PLAN-ROOT SINK
| | 92:HASH JOIN [INNER JOIN]
| | | hash predicates: ss_item_sk = iss.i_item_sk
| | | fk/pk conjuncts: ss_item_sk = iss.i_item_sk
+| | | runtime filters: RF093[min_max] <- iss.i_item_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=72,73 row-size=32B cardinality=2.88M
| | | in pipelines: 89(GETNEXT), 90(OPEN)
| | |
| | |--90:SCAN HDFS [tpcds_parquet.item iss]
| | | HDFS partitions=1/1 files=1 size=1.73MB
+| | | runtime filters: RF081[min_max] -> iss.i_brand_id, RF082[min_max] -> iss.i_category_id, RF083[min_max] -> iss.i_class_id
| | | stored statistics:
| | | table: rows=18.00K size=1.73MB
| | | columns: all
@@ -453,9 +467,10 @@ PLAN-ROOT SINK
| | | in pipelines: 90(GETNEXT)
| | |
| | 89:SCAN HDFS [tpcds_parquet.store_sales]
-| | HDFS partitions=1824/1824 files=1824 size=200.95MB
+| | HDFS partitions=1824/1824 files=1824 size=200.96MB
+| | runtime filters: RF093[min_max] -> ss_item_sk
| | stored statistics:
-| | table: rows=2.88M size=200.95MB
+| | table: rows=2.88M size=200.96MB
| | partitions: 1824/1824 rows=2.88M
| | columns: all
| | extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -466,12 +481,14 @@ PLAN-ROOT SINK
| 111:HASH JOIN [INNER JOIN]
| | hash predicates: ws_item_sk = i_item_sk
| | fk/pk conjuncts: ws_item_sk = i_item_sk
+| | runtime filters: RF075[min_max] <- i_item_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=68,70,69 row-size=52B cardinality=42.85K
| | in pipelines: 85(GETNEXT), 86(OPEN)
| |
| |--86:SCAN HDFS [tpcds_parquet.item]
| | HDFS partitions=1/1 files=1 size=1.73MB
+| | runtime filters: RF073[min_max] -> tpcds_parquet.item.i_item_sk
| | stored statistics:
| | table: rows=18.00K size=1.73MB
| | columns: all
@@ -483,7 +500,7 @@ PLAN-ROOT SINK
| 110:HASH JOIN [INNER JOIN]
| | hash predicates: ws_sold_date_sk = d_date_sk
| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| | runtime filters: RF088[bloom] <- d_date_sk
+| | runtime filters: RF076[bloom] <- d_date_sk, RF077[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=68,70 row-size=32B cardinality=42.85K
| | in pipelines: 85(GETNEXT), 87(OPEN)
@@ -503,7 +520,7 @@ PLAN-ROOT SINK
| |
| 85:SCAN HDFS [tpcds_parquet.web_sales]
| HDFS partitions=1/1 files=2 size=45.09MB
-| runtime filters: RF088[bloom] -> ws_sold_date_sk
+| runtime filters: RF077[min_max] -> ws_sold_date_sk, RF073[min_max] -> ws_item_sk, RF075[min_max] -> ws_item_sk, RF076[bloom] -> ws_sold_date_sk
| stored statistics:
| table: rows=719.38K size=45.09MB
| columns: all
@@ -532,6 +549,7 @@ PLAN-ROOT SINK
| | |--82:HASH JOIN [INNER JOIN]
| | | | hash predicates: ws_sold_date_sk = d_date_sk
| | | | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
+| | | | runtime filters: RF071[min_max] <- d_date_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=60,61 row-size=20B cardinality=719.38K
| | | | in pipelines: 80(GETNEXT), 81(OPEN)
@@ -551,6 +569,7 @@ PLAN-ROOT SINK
| | | |
| | | 80:SCAN HDFS [tpcds_parquet.web_sales]
| | | HDFS partitions=1/1 files=2 size=45.09MB
+| | | runtime filters: RF071[min_max] -> ws_sold_date_sk
| | | stored statistics:
| | | table: rows=719.38K size=45.09MB
| | | columns: all
@@ -562,6 +581,7 @@ PLAN-ROOT SINK
| | |--79:HASH JOIN [INNER JOIN]
| | | | hash predicates: cs_sold_date_sk = d_date_sk
| | | | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
+| | | | runtime filters: RF069[min_max] <- d_date_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=58,59 row-size=20B cardinality=1.44M
| | | | in pipelines: 77(GETNEXT), 78(OPEN)
@@ -581,6 +601,7 @@ PLAN-ROOT SINK
| | | |
| | | 77:SCAN HDFS [tpcds_parquet.catalog_sales]
| | | HDFS partitions=1/1 files=3 size=96.62MB
+| | | runtime filters: RF069[min_max] -> cs_sold_date_sk
| | | stored statistics:
| | | table: rows=1.44M size=96.62MB
| | | columns: all
@@ -610,9 +631,9 @@ PLAN-ROOT SINK
| | | in pipelines: 75(GETNEXT)
| | |
| | 74:SCAN HDFS [tpcds_parquet.store_sales]
-| | HDFS partitions=1824/1824 files=1824 size=200.95MB
+| | HDFS partitions=1824/1824 files=1824 size=200.96MB
| | stored statistics:
-| | table: rows=2.88M size=200.95MB
+| | table: rows=2.88M size=200.96MB
| | partitions: 1824/1824 rows=2.88M
| | columns: all
| | extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -629,6 +650,7 @@ PLAN-ROOT SINK
| |
| 71:HASH JOIN [LEFT SEMI JOIN]
| | hash predicates: cs_item_sk = tpcds_parquet.item.i_item_sk
+| | runtime filters: RF037[min_max] <- tpcds_parquet.item.i_item_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=34,36,35 row-size=52B cardinality=85.31K
| | in pipelines: 43(GETNEXT), 70(OPEN)
@@ -642,6 +664,7 @@ PLAN-ROOT SINK
| | 67:HASH JOIN [INNER JOIN]
| | | hash predicates: iss.i_brand_id = i_brand_id, iss.i_category_id = i_category_id, iss.i_class_id = i_class_id
| | | fk/pk conjuncts: iss.i_brand_id = i_brand_id, iss.i_category_id = i_category_id, iss.i_class_id = i_class_id
+| | | runtime filters: RF045[min_max] <- i_brand_id, RF046[min_max] <- i_category_id, RF047[min_max] <- i_class_id
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=41,37 row-size=32B cardinality=148.80K
| | | in pipelines: 52(GETNEXT), 46(OPEN)
@@ -671,6 +694,7 @@ PLAN-ROOT SINK
| | | 62:HASH JOIN [INNER JOIN]
| | | | hash predicates: ws_sold_date_sk = d3.d_date_sk
| | | | fk/pk conjuncts: ws_sold_date_sk = d3.d_date_sk
+| | | | runtime filters: RF063[min_max] <- d3.d_date_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=47,48,49 row-size=40B cardinality=719.38K
| | | | in pipelines: 58(GETNEXT), 60(OPEN)
@@ -691,6 +715,7 @@ PLAN-ROOT SINK
| | | 61:HASH JOIN [INNER JOIN]
| | | | hash predicates: ws_item_sk = iws.i_item_sk
| | | | fk/pk conjuncts: ws_item_sk = iws.i_item_sk
+| | | | runtime filters: RF065[min_max] <- iws.i_item_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=47,48 row-size=32B cardinality=719.38K
| | | | in pipelines: 58(GETNEXT), 59(OPEN)
@@ -707,6 +732,7 @@ PLAN-ROOT SINK
| | | |
| | | 58:SCAN HDFS [tpcds_parquet.web_sales]
| | | HDFS partitions=1/1 files=2 size=45.09MB
+| | | runtime filters: RF063[min_max] -> ws_sold_date_sk, RF065[min_max] -> ws_item_sk
| | | stored statistics:
| | | table: rows=719.38K size=45.09MB
| | | columns: all
@@ -730,6 +756,7 @@ PLAN-ROOT SINK
| | | 57:HASH JOIN [INNER JOIN]
| | | | hash predicates: cs_sold_date_sk = d2.d_date_sk
| | | | fk/pk conjuncts: cs_sold_date_sk = d2.d_date_sk
+| | | | runtime filters: RF059[min_max] <- d2.d_date_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=43,44,45 row-size=40B cardinality=1.44M
| | | | in pipelines: 53(GETNEXT), 55(OPEN)
@@ -750,6 +777,7 @@ PLAN-ROOT SINK
| | | 56:HASH JOIN [INNER JOIN]
| | | | hash predicates: cs_item_sk = ics.i_item_sk
| | | | fk/pk conjuncts: cs_item_sk = ics.i_item_sk
+| | | | runtime filters: RF061[min_max] <- ics.i_item_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=43,44 row-size=32B cardinality=1.44M
| | | | in pipelines: 53(GETNEXT), 54(OPEN)
@@ -766,6 +794,7 @@ PLAN-ROOT SINK
| | | |
| | | 53:SCAN HDFS [tpcds_parquet.catalog_sales]
| | | HDFS partitions=1/1 files=3 size=96.62MB
+| | | runtime filters: RF059[min_max] -> cs_sold_date_sk, RF061[min_max] -> cs_item_sk
| | | stored statistics:
| | | table: rows=1.44M size=96.62MB
| | | columns: all
@@ -803,12 +832,14 @@ PLAN-ROOT SINK
| | 50:HASH JOIN [INNER JOIN]
| | | hash predicates: ss_item_sk = iss.i_item_sk
| | | fk/pk conjuncts: ss_item_sk = iss.i_item_sk
+| | | runtime filters: RF057[min_max] <- iss.i_item_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=38,39 row-size=32B cardinality=2.88M
| | | in pipelines: 47(GETNEXT), 48(OPEN)
| | |
| | |--48:SCAN HDFS [tpcds_parquet.item iss]
| | | HDFS partitions=1/1 files=1 size=1.73MB
+| | | runtime filters: RF045[min_max] -> iss.i_brand_id, RF046[min_max] -> iss.i_category_id, RF047[min_max] -> iss.i_class_id
| | | stored statistics:
| | | table: rows=18.00K size=1.73MB
| | | columns: all
@@ -818,9 +849,10 @@ PLAN-ROOT SINK
| | | in pipelines: 48(GETNEXT)
| | |
| | 47:SCAN HDFS [tpcds_parquet.store_sales]
-| | HDFS partitions=1824/1824 files=1824 size=200.95MB
+| | HDFS partitions=1824/1824 files=1824 size=200.96MB
+| | runtime filters: RF057[min_max] -> ss_item_sk
| | stored statistics:
-| | table: rows=2.88M size=200.95MB
+| | table: rows=2.88M size=200.96MB
| | partitions: 1824/1824 rows=2.88M
| | columns: all
| | extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -831,12 +863,14 @@ PLAN-ROOT SINK
| 69:HASH JOIN [INNER JOIN]
| | hash predicates: cs_item_sk = i_item_sk
| | fk/pk conjuncts: cs_item_sk = i_item_sk
+| | runtime filters: RF039[min_max] <- i_item_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=34,36,35 row-size=52B cardinality=85.31K
| | in pipelines: 43(GETNEXT), 44(OPEN)
| |
| |--44:SCAN HDFS [tpcds_parquet.item]
| | HDFS partitions=1/1 files=1 size=1.73MB
+| | runtime filters: RF037[min_max] -> tpcds_parquet.item.i_item_sk
| | stored statistics:
| | table: rows=18.00K size=1.73MB
| | columns: all
@@ -848,7 +882,7 @@ PLAN-ROOT SINK
| 68:HASH JOIN [INNER JOIN]
| | hash predicates: cs_sold_date_sk = d_date_sk
| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
-| | runtime filters: RF046[bloom] <- d_date_sk
+| | runtime filters: RF040[bloom] <- d_date_sk, RF041[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=34,36 row-size=32B cardinality=85.31K
| | in pipelines: 43(GETNEXT), 45(OPEN)
@@ -868,7 +902,7 @@ PLAN-ROOT SINK
| |
| 43:SCAN HDFS [tpcds_parquet.catalog_sales]
| HDFS partitions=1/1 files=3 size=96.62MB
-| runtime filters: RF046[bloom] -> cs_sold_date_sk
+| runtime filters: RF041[min_max] -> cs_sold_date_sk, RF037[min_max] -> cs_item_sk, RF039[min_max] -> cs_item_sk, RF040[bloom] -> cs_sold_date_sk
| stored statistics:
| table: rows=1.44M size=96.62MB
| columns: all
@@ -897,6 +931,7 @@ PLAN-ROOT SINK
| |--40:HASH JOIN [INNER JOIN]
| | | hash predicates: ws_sold_date_sk = d_date_sk
| | | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
+| | | runtime filters: RF035[min_max] <- d_date_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=26,27 row-size=20B cardinality=719.38K
| | | in pipelines: 38(GETNEXT), 39(OPEN)
@@ -916,6 +951,7 @@ PLAN-ROOT SINK
| | |
| | 38:SCAN HDFS [tpcds_parquet.web_sales]
| | HDFS partitions=1/1 files=2 size=45.09MB
+| | runtime filters: RF035[min_max] -> ws_sold_date_sk
| | stored statistics:
| | table: rows=719.38K size=45.09MB
| | columns: all
@@ -927,6 +963,7 @@ PLAN-ROOT SINK
| |--37:HASH JOIN [INNER JOIN]
| | | hash predicates: cs_sold_date_sk = d_date_sk
| | | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
+| | | runtime filters: RF033[min_max] <- d_date_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=24,25 row-size=20B cardinality=1.44M
| | | in pipelines: 35(GETNEXT), 36(OPEN)
@@ -946,6 +983,7 @@ PLAN-ROOT SINK
| | |
| | 35:SCAN HDFS [tpcds_parquet.catalog_sales]
| | HDFS partitions=1/1 files=3 size=96.62MB
+| | runtime filters: RF033[min_max] -> cs_sold_date_sk
| | stored statistics:
| | table: rows=1.44M size=96.62MB
| | columns: all
@@ -975,9 +1013,9 @@ PLAN-ROOT SINK
| | in pipelines: 33(GETNEXT)
| |
| 32:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -994,7 +1032,7 @@ PLAN-ROOT SINK
|
29:HASH JOIN [LEFT SEMI JOIN]
| hash predicates: ss_item_sk = tpcds_parquet.item.i_item_sk
-| runtime filters: RF000[bloom] <- tpcds_parquet.item.i_item_sk
+| runtime filters: RF000[bloom] <- tpcds_parquet.item.i_item_sk, RF001[min_max] <- tpcds_parquet.item.i_item_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=0,2,1 row-size=52B cardinality=170.55K
| in pipelines: 01(GETNEXT), 28(OPEN)
@@ -1008,7 +1046,7 @@ PLAN-ROOT SINK
| 25:HASH JOIN [INNER JOIN]
| | hash predicates: iss.i_brand_id = i_brand_id, iss.i_category_id = i_category_id, iss.i_class_id = i_class_id
| | fk/pk conjuncts: iss.i_brand_id = i_brand_id, iss.i_category_id = i_category_id, iss.i_class_id = i_class_id
-| | runtime filters: RF006[bloom] <- i_brand_id, RF007[bloom] <- i_category_id, RF008[bloom] <- i_class_id
+| | runtime filters: RF006[bloom] <- i_brand_id, RF007[bloom] <- i_category_id, RF008[bloom] <- i_class_id, RF009[min_max] <- i_brand_id, RF010[min_max] <- i_category_id, RF011[min_max] <- i_class_id
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=7,3 row-size=32B cardinality=148.80K
| | in pipelines: 10(GETNEXT), 04(OPEN)
@@ -1039,6 +1077,7 @@ PLAN-ROOT SINK
| | 20:HASH JOIN [INNER JOIN]
| | | hash predicates: ws_sold_date_sk = d3.d_date_sk
| | | fk/pk conjuncts: ws_sold_date_sk = d3.d_date_sk
+| | | runtime filters: RF027[min_max] <- d3.d_date_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=13,14,15 row-size=40B cardinality=719.38K
| | | in pipelines: 16(GETNEXT), 18(OPEN)
@@ -1059,6 +1098,7 @@ PLAN-ROOT SINK
| | 19:HASH JOIN [INNER JOIN]
| | | hash predicates: ws_item_sk = iws.i_item_sk
| | | fk/pk conjuncts: ws_item_sk = iws.i_item_sk
+| | | runtime filters: RF029[min_max] <- iws.i_item_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=13,14 row-size=32B cardinality=719.38K
| | | in pipelines: 16(GETNEXT), 17(OPEN)
@@ -1075,6 +1115,7 @@ PLAN-ROOT SINK
| | |
| | 16:SCAN HDFS [tpcds_parquet.web_sales]
| | HDFS partitions=1/1 files=2 size=45.09MB
+| | runtime filters: RF027[min_max] -> ws_sold_date_sk, RF029[min_max] -> ws_item_sk
| | stored statistics:
| | table: rows=719.38K size=45.09MB
| | columns: all
@@ -1098,6 +1139,7 @@ PLAN-ROOT SINK
| | 15:HASH JOIN [INNER JOIN]
| | | hash predicates: cs_sold_date_sk = d2.d_date_sk
| | | fk/pk conjuncts: cs_sold_date_sk = d2.d_date_sk
+| | | runtime filters: RF023[min_max] <- d2.d_date_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=9,10,11 row-size=40B cardinality=1.44M
| | | in pipelines: 11(GETNEXT), 13(OPEN)
@@ -1118,6 +1160,7 @@ PLAN-ROOT SINK
| | 14:HASH JOIN [INNER JOIN]
| | | hash predicates: cs_item_sk = ics.i_item_sk
| | | fk/pk conjuncts: cs_item_sk = ics.i_item_sk
+| | | runtime filters: RF025[min_max] <- ics.i_item_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=9,10 row-size=32B cardinality=1.44M
| | | in pipelines: 11(GETNEXT), 12(OPEN)
@@ -1134,6 +1177,7 @@ PLAN-ROOT SINK
| | |
| | 11:SCAN HDFS [tpcds_parquet.catalog_sales]
| | HDFS partitions=1/1 files=3 size=96.62MB
+| | runtime filters: RF023[min_max] -> cs_sold_date_sk, RF025[min_max] -> cs_item_sk
| | stored statistics:
| | table: rows=1.44M size=96.62MB
| | columns: all
@@ -1171,13 +1215,14 @@ PLAN-ROOT SINK
| 08:HASH JOIN [INNER JOIN]
| | hash predicates: ss_item_sk = iss.i_item_sk
| | fk/pk conjuncts: ss_item_sk = iss.i_item_sk
+| | runtime filters: RF021[min_max] <- iss.i_item_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=4,5 row-size=32B cardinality=2.88M
| | in pipelines: 05(GETNEXT), 06(OPEN)
| |
| |--06:SCAN HDFS [tpcds_parquet.item iss]
| | HDFS partitions=1/1 files=1 size=1.73MB
-| | runtime filters: RF006[bloom] -> iss.i_brand_id, RF007[bloom] -> iss.i_category_id, RF008[bloom] -> iss.i_class_id, RF012[bloom] -> iss.i_brand_id, RF013[bloom] -> iss.i_category_id
+| | runtime filters: RF009[min_max] -> iss.i_brand_id, RF010[min_max] -> iss.i_category_id, RF011[min_max] -> iss.i_class_id, RF006[bloom] -> iss.i_brand_id, RF007[bloom] -> iss.i_category_id, RF008[bloom] -> iss.i_class_id, RF012[bloom] -> iss.i_brand_id, RF013[bloom] -> iss.i_category_id
| | stored statistics:
| | table: rows=18.00K size=1.73MB
| | columns: all
@@ -1187,9 +1232,10 @@ PLAN-ROOT SINK
| | in pipelines: 06(GETNEXT)
| |
| 05:SCAN HDFS [tpcds_parquet.store_sales]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
+| runtime filters: RF021[min_max] -> ss_item_sk
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1200,14 +1246,14 @@ PLAN-ROOT SINK
27:HASH JOIN [INNER JOIN]
| hash predicates: ss_item_sk = i_item_sk
| fk/pk conjuncts: ss_item_sk = i_item_sk
-| runtime filters: RF002[bloom] <- i_item_sk
+| runtime filters: RF002[bloom] <- i_item_sk, RF003[min_max] <- i_item_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=0,2,1 row-size=52B cardinality=170.55K
| in pipelines: 01(GETNEXT), 02(OPEN)
|
|--02:SCAN HDFS [tpcds_parquet.item]
| HDFS partitions=1/1 files=1 size=1.73MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.item.i_item_sk
+| runtime filters: RF001[min_max] -> tpcds_parquet.item.i_item_sk, RF000[bloom] -> tpcds_parquet.item.i_item_sk
| stored statistics:
| table: rows=18.00K size=1.73MB
| columns: all
@@ -1238,10 +1284,10 @@ PLAN-ROOT SINK
| in pipelines: 03(GETNEXT)
|
01:SCAN HDFS [tpcds_parquet.store_sales]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF004[bloom] -> ss_sold_date_sk, RF000[bloom] -> ss_item_sk, RF002[bloom] -> ss_item_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF001[min_max] -> ss_item_sk, RF003[min_max] -> ss_item_sk, RF004[bloom] -> ss_sold_date_sk, RF000[bloom] -> ss_item_sk, RF002[bloom] -> ss_item_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1371,6 +1417,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | |--124:HASH JOIN [INNER JOIN, BROADCAST]
| | | | hash predicates: ws_sold_date_sk = d_date_sk
| | | | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
+| | | | runtime filters: RF107[min_max] <- d_date_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=94,95 row-size=20B cardinality=719.38K
| | | | in pipelines: 122(GETNEXT), 123(OPEN)
@@ -1397,6 +1444,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | |
| | | 122:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| | | HDFS partitions=1/1 files=2 size=45.09MB
+| | | runtime filters: RF107[min_max] -> ws_sold_date_sk
| | | stored statistics:
| | | table: rows=719.38K size=45.09MB
| | | columns: all
@@ -1408,6 +1456,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | |--121:HASH JOIN [INNER JOIN, BROADCAST]
| | | | hash predicates: cs_sold_date_sk = d_date_sk
| | | | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
+| | | | runtime filters: RF105[min_max] <- d_date_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=92,93 row-size=20B cardinality=1.44M
| | | | in pipelines: 119(GETNEXT), 120(OPEN)
@@ -1434,6 +1483,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | |
| | | 119:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| | | HDFS partitions=1/1 files=3 size=96.62MB
+| | | runtime filters: RF105[min_max] -> cs_sold_date_sk
| | | stored statistics:
| | | table: rows=1.44M size=96.62MB
| | | columns: all
@@ -1470,9 +1520,9 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | in pipelines: 117(GETNEXT)
| | |
| | 116:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| | HDFS partitions=1824/1824 files=1824 size=200.95MB
+| | HDFS partitions=1824/1824 files=1824 size=200.96MB
| | stored statistics:
-| | table: rows=2.88M size=200.95MB
+| | table: rows=2.88M size=200.96MB
| | partitions: 1824/1824 rows=2.88M
| | columns: all
| | extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1503,6 +1553,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| |
| 113:HASH JOIN [LEFT SEMI JOIN, BROADCAST]
| | hash predicates: ws_item_sk = tpcds_parquet.item.i_item_sk
+| | runtime filters: RF073[min_max] <- tpcds_parquet.item.i_item_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=68,70,69 row-size=52B cardinality=42.85K
| | in pipelines: 85(GETNEXT), 204(OPEN)
@@ -1536,6 +1587,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | 109:HASH JOIN [INNER JOIN, PARTITIONED]
| | | hash predicates: iss.i_brand_id = i_brand_id, iss.i_category_id = i_category_id, iss.i_class_id = i_class_id
| | | fk/pk conjuncts: iss.i_brand_id = i_brand_id, iss.i_category_id = i_category_id, iss.i_class_id = i_class_id
+| | | runtime filters: RF081[min_max] <- i_brand_id, RF082[min_max] <- i_category_id, RF083[min_max] <- i_class_id
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=75,71 row-size=32B cardinality=148.80K
| | | in pipelines: 191(GETNEXT), 88(OPEN)
@@ -1592,6 +1644,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | 104:HASH JOIN [INNER JOIN, BROADCAST]
| | | | hash predicates: ws_sold_date_sk = d3.d_date_sk
| | | | fk/pk conjuncts: ws_sold_date_sk = d3.d_date_sk
+| | | | runtime filters: RF099[min_max] <- d3.d_date_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=81,82,83 row-size=40B cardinality=719.38K
| | | | in pipelines: 100(GETNEXT), 102(OPEN)
@@ -1619,6 +1672,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | 103:HASH JOIN [INNER JOIN, BROADCAST]
| | | | hash predicates: ws_item_sk = iws.i_item_sk
| | | | fk/pk conjuncts: ws_item_sk = iws.i_item_sk
+| | | | runtime filters: RF101[min_max] <- iws.i_item_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=81,82 row-size=32B cardinality=719.38K
| | | | in pipelines: 100(GETNEXT), 101(OPEN)
@@ -1642,6 +1696,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | |
| | | 100:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| | | HDFS partitions=1/1 files=2 size=45.09MB
+| | | runtime filters: RF099[min_max] -> ws_sold_date_sk, RF101[min_max] -> ws_item_sk
| | | stored statistics:
| | | table: rows=719.38K size=45.09MB
| | | columns: all
@@ -1685,6 +1740,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | 99:HASH JOIN [INNER JOIN, BROADCAST]
| | | | hash predicates: cs_sold_date_sk = d2.d_date_sk
| | | | fk/pk conjuncts: cs_sold_date_sk = d2.d_date_sk
+| | | | runtime filters: RF095[min_max] <- d2.d_date_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=77,78,79 row-size=40B cardinality=1.44M
| | | | in pipelines: 95(GETNEXT), 97(OPEN)
@@ -1712,6 +1768,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | 98:HASH JOIN [INNER JOIN, BROADCAST]
| | | | hash predicates: cs_item_sk = ics.i_item_sk
| | | | fk/pk conjuncts: cs_item_sk = ics.i_item_sk
+| | | | runtime filters: RF097[min_max] <- ics.i_item_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=77,78 row-size=32B cardinality=1.44M
| | | | in pipelines: 95(GETNEXT), 96(OPEN)
@@ -1735,6 +1792,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | |
| | | 95:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| | | HDFS partitions=1/1 files=3 size=96.62MB
+| | | runtime filters: RF095[min_max] -> cs_sold_date_sk, RF097[min_max] -> cs_item_sk
| | | stored statistics:
| | | table: rows=1.44M size=96.62MB
| | | columns: all
@@ -1792,6 +1850,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | 92:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash predicates: ss_item_sk = iss.i_item_sk
| | | fk/pk conjuncts: ss_item_sk = iss.i_item_sk
+| | | runtime filters: RF093[min_max] <- iss.i_item_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=72,73 row-size=32B cardinality=2.88M
| | | in pipelines: 89(GETNEXT), 90(OPEN)
@@ -1805,6 +1864,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | Per-Host Resources: mem-estimate=64.00MB mem-reservation=256.00KB thread-reservation=2
| | | 90:SCAN HDFS [tpcds_parquet.item iss, RANDOM]
| | | HDFS partitions=1/1 files=1 size=1.73MB
+| | | runtime filters: RF081[min_max] -> iss.i_brand_id, RF082[min_max] -> iss.i_category_id, RF083[min_max] -> iss.i_class_id
| | | stored statistics:
| | | table: rows=18.00K size=1.73MB
| | | columns: all
@@ -1814,9 +1874,10 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | in pipelines: 90(GETNEXT)
| | |
| | 89:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| | HDFS partitions=1824/1824 files=1824 size=200.95MB
+| | HDFS partitions=1824/1824 files=1824 size=200.96MB
+| | runtime filters: RF093[min_max] -> ss_item_sk
| | stored statistics:
-| | table: rows=2.88M size=200.95MB
+| | table: rows=2.88M size=200.96MB
| | partitions: 1824/1824 rows=2.88M
| | columns: all
| | extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -1827,6 +1888,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| 111:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: ws_item_sk = i_item_sk
| | fk/pk conjuncts: ws_item_sk = i_item_sk
+| | runtime filters: RF075[min_max] <- i_item_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=68,70,69 row-size=52B cardinality=42.85K
| | in pipelines: 85(GETNEXT), 86(OPEN)
@@ -1840,6 +1902,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | Per-Host Resources: mem-estimate=64.00MB mem-reservation=256.00KB thread-reservation=2
| | 86:SCAN HDFS [tpcds_parquet.item, RANDOM]
| | HDFS partitions=1/1 files=1 size=1.73MB
+| | runtime filters: RF073[min_max] -> tpcds_parquet.item.i_item_sk
| | stored statistics:
| | table: rows=18.00K size=1.73MB
| | columns: all
@@ -1851,7 +1914,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| 110:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: ws_sold_date_sk = d_date_sk
| | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
-| | runtime filters: RF088[bloom] <- d_date_sk
+| | runtime filters: RF076[bloom] <- d_date_sk, RF077[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=68,70 row-size=32B cardinality=42.85K
| | in pipelines: 85(GETNEXT), 87(OPEN)
@@ -1878,7 +1941,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| |
| 85:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| HDFS partitions=1/1 files=2 size=45.09MB
-| runtime filters: RF088[bloom] -> ws_sold_date_sk
+| runtime filters: RF077[min_max] -> ws_sold_date_sk, RF073[min_max] -> ws_item_sk, RF075[min_max] -> ws_item_sk, RF076[bloom] -> ws_sold_date_sk
| stored statistics:
| table: rows=719.38K size=45.09MB
| columns: all
@@ -1927,6 +1990,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | |--82:HASH JOIN [INNER JOIN, BROADCAST]
| | | | hash predicates: ws_sold_date_sk = d_date_sk
| | | | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
+| | | | runtime filters: RF071[min_max] <- d_date_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=60,61 row-size=20B cardinality=719.38K
| | | | in pipelines: 80(GETNEXT), 81(OPEN)
@@ -1953,6 +2017,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | |
| | | 80:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| | | HDFS partitions=1/1 files=2 size=45.09MB
+| | | runtime filters: RF071[min_max] -> ws_sold_date_sk
| | | stored statistics:
| | | table: rows=719.38K size=45.09MB
| | | columns: all
@@ -1964,6 +2029,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | |--79:HASH JOIN [INNER JOIN, BROADCAST]
| | | | hash predicates: cs_sold_date_sk = d_date_sk
| | | | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
+| | | | runtime filters: RF069[min_max] <- d_date_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=58,59 row-size=20B cardinality=1.44M
| | | | in pipelines: 77(GETNEXT), 78(OPEN)
@@ -1990,6 +2056,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | |
| | | 77:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| | | HDFS partitions=1/1 files=3 size=96.62MB
+| | | runtime filters: RF069[min_max] -> cs_sold_date_sk
| | | stored statistics:
| | | table: rows=1.44M size=96.62MB
| | | columns: all
@@ -2026,9 +2093,9 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | in pipelines: 75(GETNEXT)
| | |
| | 74:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| | HDFS partitions=1824/1824 files=1824 size=200.95MB
+| | HDFS partitions=1824/1824 files=1824 size=200.96MB
| | stored statistics:
-| | table: rows=2.88M size=200.95MB
+| | table: rows=2.88M size=200.96MB
| | partitions: 1824/1824 rows=2.88M
| | columns: all
| | extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -2059,6 +2126,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| |
| 71:HASH JOIN [LEFT SEMI JOIN, BROADCAST]
| | hash predicates: cs_item_sk = tpcds_parquet.item.i_item_sk
+| | runtime filters: RF037[min_max] <- tpcds_parquet.item.i_item_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=34,36,35 row-size=52B cardinality=85.31K
| | in pipelines: 43(GETNEXT), 176(OPEN)
@@ -2092,6 +2160,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | 67:HASH JOIN [INNER JOIN, PARTITIONED]
| | | hash predicates: iss.i_brand_id = i_brand_id, iss.i_category_id = i_category_id, iss.i_class_id = i_class_id
| | | fk/pk conjuncts: iss.i_brand_id = i_brand_id, iss.i_category_id = i_category_id, iss.i_class_id = i_class_id
+| | | runtime filters: RF045[min_max] <- i_brand_id, RF046[min_max] <- i_category_id, RF047[min_max] <- i_class_id
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=41,37 row-size=32B cardinality=148.80K
| | | in pipelines: 163(GETNEXT), 46(OPEN)
@@ -2148,6 +2217,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | 62:HASH JOIN [INNER JOIN, BROADCAST]
| | | | hash predicates: ws_sold_date_sk = d3.d_date_sk
| | | | fk/pk conjuncts: ws_sold_date_sk = d3.d_date_sk
+| | | | runtime filters: RF063[min_max] <- d3.d_date_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=47,48,49 row-size=40B cardinality=719.38K
| | | | in pipelines: 58(GETNEXT), 60(OPEN)
@@ -2175,6 +2245,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | 61:HASH JOIN [INNER JOIN, BROADCAST]
| | | | hash predicates: ws_item_sk = iws.i_item_sk
| | | | fk/pk conjuncts: ws_item_sk = iws.i_item_sk
+| | | | runtime filters: RF065[min_max] <- iws.i_item_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=47,48 row-size=32B cardinality=719.38K
| | | | in pipelines: 58(GETNEXT), 59(OPEN)
@@ -2198,6 +2269,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | |
| | | 58:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| | | HDFS partitions=1/1 files=2 size=45.09MB
+| | | runtime filters: RF063[min_max] -> ws_sold_date_sk, RF065[min_max] -> ws_item_sk
| | | stored statistics:
| | | table: rows=719.38K size=45.09MB
| | | columns: all
@@ -2241,6 +2313,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | 57:HASH JOIN [INNER JOIN, BROADCAST]
| | | | hash predicates: cs_sold_date_sk = d2.d_date_sk
| | | | fk/pk conjuncts: cs_sold_date_sk = d2.d_date_sk
+| | | | runtime filters: RF059[min_max] <- d2.d_date_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=43,44,45 row-size=40B cardinality=1.44M
| | | | in pipelines: 53(GETNEXT), 55(OPEN)
@@ -2268,6 +2341,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | 56:HASH JOIN [INNER JOIN, BROADCAST]
| | | | hash predicates: cs_item_sk = ics.i_item_sk
| | | | fk/pk conjuncts: cs_item_sk = ics.i_item_sk
+| | | | runtime filters: RF061[min_max] <- ics.i_item_sk
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | | tuple-ids=43,44 row-size=32B cardinality=1.44M
| | | | in pipelines: 53(GETNEXT), 54(OPEN)
@@ -2291,6 +2365,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | |
| | | 53:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| | | HDFS partitions=1/1 files=3 size=96.62MB
+| | | runtime filters: RF059[min_max] -> cs_sold_date_sk, RF061[min_max] -> cs_item_sk
| | | stored statistics:
| | | table: rows=1.44M size=96.62MB
| | | columns: all
@@ -2348,6 +2423,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | 50:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash predicates: ss_item_sk = iss.i_item_sk
| | | fk/pk conjuncts: ss_item_sk = iss.i_item_sk
+| | | runtime filters: RF057[min_max] <- iss.i_item_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=38,39 row-size=32B cardinality=2.88M
| | | in pipelines: 47(GETNEXT), 48(OPEN)
@@ -2361,6 +2437,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | Per-Host Resources: mem-estimate=64.00MB mem-reservation=256.00KB thread-reservation=2
| | | 48:SCAN HDFS [tpcds_parquet.item iss, RANDOM]
| | | HDFS partitions=1/1 files=1 size=1.73MB
+| | | runtime filters: RF045[min_max] -> iss.i_brand_id, RF046[min_max] -> iss.i_category_id, RF047[min_max] -> iss.i_class_id
| | | stored statistics:
| | | table: rows=18.00K size=1.73MB
| | | columns: all
@@ -2370,9 +2447,10 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | | in pipelines: 48(GETNEXT)
| | |
| | 47:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| | HDFS partitions=1824/1824 files=1824 size=200.95MB
+| | HDFS partitions=1824/1824 files=1824 size=200.96MB
+| | runtime filters: RF057[min_max] -> ss_item_sk
| | stored statistics:
-| | table: rows=2.88M size=200.95MB
+| | table: rows=2.88M size=200.96MB
| | partitions: 1824/1824 rows=2.88M
| | columns: all
| | extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -2383,6 +2461,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| 69:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: cs_item_sk = i_item_sk
| | fk/pk conjuncts: cs_item_sk = i_item_sk
+| | runtime filters: RF039[min_max] <- i_item_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=34,36,35 row-size=52B cardinality=85.31K
| | in pipelines: 43(GETNEXT), 44(OPEN)
@@ -2396,6 +2475,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | Per-Host Resources: mem-estimate=64.00MB mem-reservation=256.00KB thread-reservation=2
| | 44:SCAN HDFS [tpcds_parquet.item, RANDOM]
| | HDFS partitions=1/1 files=1 size=1.73MB
+| | runtime filters: RF037[min_max] -> tpcds_parquet.item.i_item_sk
| | stored statistics:
| | table: rows=18.00K size=1.73MB
| | columns: all
@@ -2407,7 +2487,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| 68:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: cs_sold_date_sk = d_date_sk
| | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
-| | runtime filters: RF046[bloom] <- d_date_sk
+| | runtime filters: RF040[bloom] <- d_date_sk, RF041[min_max] <- d_date_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=34,36 row-size=32B cardinality=85.31K
| | in pipelines: 43(GETNEXT), 45(OPEN)
@@ -2434,7 +2514,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| |
| 43:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| HDFS partitions=1/1 files=3 size=96.62MB
-| runtime filters: RF046[bloom] -> cs_sold_date_sk
+| runtime filters: RF041[min_max] -> cs_sold_date_sk, RF037[min_max] -> cs_item_sk, RF039[min_max] -> cs_item_sk, RF040[bloom] -> cs_sold_date_sk
| stored statistics:
| table: rows=1.44M size=96.62MB
| columns: all
@@ -2483,6 +2563,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| |--40:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash predicates: ws_sold_date_sk = d_date_sk
| | | fk/pk conjuncts: ws_sold_date_sk = d_date_sk
+| | | runtime filters: RF035[min_max] <- d_date_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=26,27 row-size=20B cardinality=719.38K
| | | in pipelines: 38(GETNEXT), 39(OPEN)
@@ -2509,6 +2590,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | |
| | 38:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| | HDFS partitions=1/1 files=2 size=45.09MB
+| | runtime filters: RF035[min_max] -> ws_sold_date_sk
| | stored statistics:
| | table: rows=719.38K size=45.09MB
| | columns: all
@@ -2520,6 +2602,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| |--37:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash predicates: cs_sold_date_sk = d_date_sk
| | | fk/pk conjuncts: cs_sold_date_sk = d_date_sk
+| | | runtime filters: RF033[min_max] <- d_date_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=24,25 row-size=20B cardinality=1.44M
| | | in pipelines: 35(GETNEXT), 36(OPEN)
@@ -2546,6 +2629,7 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | |
| | 35:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| | HDFS partitions=1/1 files=3 size=96.62MB
+| | runtime filters: RF033[min_max] -> cs_sold_date_sk
| | stored statistics:
| | table: rows=1.44M size=96.62MB
| | columns: all
@@ -2582,9 +2666,9 @@ Per-Host Resources: mem-estimate=111.57MB mem-reservation=79.75MB thread-reserva
| | in pipelines: 33(GETNEXT)
| |
| 32:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -2615,7 +2699,7 @@ Per-Host Resources: mem-estimate=67.37MB mem-reservation=14.81MB thread-reservat
|
29:HASH JOIN [LEFT SEMI JOIN, BROADCAST]
| hash predicates: ss_item_sk = tpcds_parquet.item.i_item_sk
-| runtime filters: RF000[bloom] <- tpcds_parquet.item.i_item_sk
+| runtime filters: RF000[bloom] <- tpcds_parquet.item.i_item_sk, RF001[min_max] <- tpcds_parquet.item.i_item_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=0,2,1 row-size=52B cardinality=170.55K
| in pipelines: 01(GETNEXT), 148(OPEN)
@@ -2649,7 +2733,7 @@ Per-Host Resources: mem-estimate=67.37MB mem-reservation=14.81MB thread-reservat
| 25:HASH JOIN [INNER JOIN, PARTITIONED]
| | hash predicates: iss.i_brand_id = i_brand_id, iss.i_category_id = i_category_id, iss.i_class_id = i_class_id
| | fk/pk conjuncts: iss.i_brand_id = i_brand_id, iss.i_category_id = i_category_id, iss.i_class_id = i_class_id
-| | runtime filters: RF006[bloom] <- i_brand_id, RF007[bloom] <- i_category_id, RF008[bloom] <- i_class_id
+| | runtime filters: RF006[bloom] <- i_brand_id, RF007[bloom] <- i_category_id, RF008[bloom] <- i_class_id, RF009[min_max] <- i_brand_id, RF010[min_max] <- i_category_id, RF011[min_max] <- i_class_id
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=7,3 row-size=32B cardinality=148.80K
| | in pipelines: 135(GETNEXT), 04(OPEN)
@@ -2707,6 +2791,7 @@ Per-Host Resources: mem-estimate=67.37MB mem-reservation=14.81MB thread-reservat
| | 20:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash predicates: ws_sold_date_sk = d3.d_date_sk
| | | fk/pk conjuncts: ws_sold_date_sk = d3.d_date_sk
+| | | runtime filters: RF027[min_max] <- d3.d_date_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=13,14,15 row-size=40B cardinality=719.38K
| | | in pipelines: 16(GETNEXT), 18(OPEN)
@@ -2734,6 +2819,7 @@ Per-Host Resources: mem-estimate=67.37MB mem-reservation=14.81MB thread-reservat
| | 19:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash predicates: ws_item_sk = iws.i_item_sk
| | | fk/pk conjuncts: ws_item_sk = iws.i_item_sk
+| | | runtime filters: RF029[min_max] <- iws.i_item_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=13,14 row-size=32B cardinality=719.38K
| | | in pipelines: 16(GETNEXT), 17(OPEN)
@@ -2757,6 +2843,7 @@ Per-Host Resources: mem-estimate=67.37MB mem-reservation=14.81MB thread-reservat
| | |
| | 16:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| | HDFS partitions=1/1 files=2 size=45.09MB
+| | runtime filters: RF027[min_max] -> ws_sold_date_sk, RF029[min_max] -> ws_item_sk
| | stored statistics:
| | table: rows=719.38K size=45.09MB
| | columns: all
@@ -2800,6 +2887,7 @@ Per-Host Resources: mem-estimate=67.37MB mem-reservation=14.81MB thread-reservat
| | 15:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash predicates: cs_sold_date_sk = d2.d_date_sk
| | | fk/pk conjuncts: cs_sold_date_sk = d2.d_date_sk
+| | | runtime filters: RF023[min_max] <- d2.d_date_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=9,10,11 row-size=40B cardinality=1.44M
| | | in pipelines: 11(GETNEXT), 13(OPEN)
@@ -2827,6 +2915,7 @@ Per-Host Resources: mem-estimate=67.37MB mem-reservation=14.81MB thread-reservat
| | 14:HASH JOIN [INNER JOIN, BROADCAST]
| | | hash predicates: cs_item_sk = ics.i_item_sk
| | | fk/pk conjuncts: cs_item_sk = ics.i_item_sk
+| | | runtime filters: RF025[min_max] <- ics.i_item_sk
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=9,10 row-size=32B cardinality=1.44M
| | | in pipelines: 11(GETNEXT), 12(OPEN)
@@ -2850,6 +2939,7 @@ Per-Host Resources: mem-estimate=67.37MB mem-reservation=14.81MB thread-reservat
| | |
| | 11:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| | HDFS partitions=1/1 files=3 size=96.62MB
+| | runtime filters: RF023[min_max] -> cs_sold_date_sk, RF025[min_max] -> cs_item_sk
| | stored statistics:
| | table: rows=1.44M size=96.62MB
| | columns: all
@@ -2907,6 +2997,7 @@ Per-Host Resources: mem-estimate=67.37MB mem-reservation=14.81MB thread-reservat
| 08:HASH JOIN [INNER JOIN, BROADCAST]
| | hash predicates: ss_item_sk = iss.i_item_sk
| | fk/pk conjuncts: ss_item_sk = iss.i_item_sk
+| | runtime filters: RF021[min_max] <- iss.i_item_sk
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=4,5 row-size=32B cardinality=2.88M
| | in pipelines: 05(GETNEXT), 06(OPEN)
@@ -2920,7 +3011,7 @@ Per-Host Resources: mem-estimate=67.37MB mem-reservation=14.81MB thread-reservat
| | Per-Host Resources: mem-estimate=69.00MB mem-reservation=5.25MB thread-reservation=2 runtime-filters-memory=5.00MB
| | 06:SCAN HDFS [tpcds_parquet.item iss, RANDOM]
| | HDFS partitions=1/1 files=1 size=1.73MB
-| | runtime filters: RF006[bloom] -> iss.i_brand_id, RF007[bloom] -> iss.i_category_id, RF008[bloom] -> iss.i_class_id, RF012[bloom] -> iss.i_brand_id, RF013[bloom] -> iss.i_category_id
+| | runtime filters: RF009[min_max] -> iss.i_brand_id, RF010[min_max] -> iss.i_category_id, RF011[min_max] -> iss.i_class_id, RF006[bloom] -> iss.i_brand_id, RF007[bloom] -> iss.i_category_id, RF008[bloom] -> iss.i_class_id, RF012[bloom] -> iss.i_brand_id, RF013[bloom] -> iss.i_category_id
| | stored statistics:
| | table: rows=18.00K size=1.73MB
| | columns: all
@@ -2930,9 +3021,10 @@ Per-Host Resources: mem-estimate=67.37MB mem-reservation=14.81MB thread-reservat
| | in pipelines: 06(GETNEXT)
| |
| 05:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
+| runtime filters: RF021[min_max] -> ss_item_sk
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -2943,7 +3035,7 @@ Per-Host Resources: mem-estimate=67.37MB mem-reservation=14.81MB thread-reservat
27:HASH JOIN [INNER JOIN, BROADCAST]
| hash predicates: ss_item_sk = i_item_sk
| fk/pk conjuncts: ss_item_sk = i_item_sk
-| runtime filters: RF002[bloom] <- i_item_sk
+| runtime filters: RF002[bloom] <- i_item_sk, RF003[min_max] <- i_item_sk
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=0,2,1 row-size=52B cardinality=170.55K
| in pipelines: 01(GETNEXT), 02(OPEN)
@@ -2957,7 +3049,7 @@ Per-Host Resources: mem-estimate=67.37MB mem-reservation=14.81MB thread-reservat
| Per-Host Resources: mem-estimate=65.00MB mem-reservation=1.25MB thread-reservation=2 runtime-filters-memory=1.00MB
| 02:SCAN HDFS [tpcds_parquet.item, RANDOM]
| HDFS partitions=1/1 files=1 size=1.73MB
-| runtime filters: RF000[bloom] -> tpcds_parquet.item.i_item_sk
+| runtime filters: RF001[min_max] -> tpcds_parquet.item.i_item_sk, RF000[bloom] -> tpcds_parquet.item.i_item_sk
| stored statistics:
| table: rows=18.00K size=1.73MB
| columns: all
@@ -2995,10 +3087,10 @@ Per-Host Resources: mem-estimate=67.37MB mem-reservation=14.81MB thread-reservat
| in pipelines: 03(GETNEXT)
|
01:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
- HDFS partitions=1824/1824 files=1824 size=200.95MB
- runtime filters: RF004[bloom] -> ss_sold_date_sk, RF000[bloom] -> ss_item_sk, RF002[bloom] -> ss_item_sk
+ HDFS partitions=1824/1824 files=1824 size=200.96MB
+ runtime filters: RF001[min_max] -> ss_item_sk, RF003[min_max] -> ss_item_sk, RF004[bloom] -> ss_sold_date_sk, RF000[bloom] -> ss_item_sk, RF002[bloom] -> ss_item_sk
stored statistics:
- table: rows=2.88M size=200.95MB
+ table: rows=2.88M size=200.96MB
partitions: 1824/1824 rows=2.88M
columns: all
extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -3145,6 +3237,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | | JOIN BUILD
| | | | | join-table-id=11 plan-id=12 cohort-id=04
| | | | | build expressions: d_date_sk
+| | | | | runtime filters: RF107[min_max] <- d_date_sk
| | | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | | |
| | | | 210:EXCHANGE [BROADCAST]
@@ -3169,6 +3262,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | |
| | | 122:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| | | HDFS partitions=1/1 files=2 size=45.09MB
+| | | runtime filters: RF107[min_max] -> ws_sold_date_sk
| | | stored statistics:
| | | table: rows=719.38K size=45.09MB
| | | columns: all
@@ -3190,6 +3284,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | | JOIN BUILD
| | | | | join-table-id=10 plan-id=11 cohort-id=04
| | | | | build expressions: d_date_sk
+| | | | | runtime filters: RF105[min_max] <- d_date_sk
| | | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | | |
| | | | 209:EXCHANGE [BROADCAST]
@@ -3214,6 +3309,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | |
| | | 119:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| | | HDFS partitions=1/1 files=3 size=96.62MB
+| | | runtime filters: RF105[min_max] -> cs_sold_date_sk
| | | stored statistics:
| | | table: rows=1.44M size=96.62MB
| | | columns: all
@@ -3258,9 +3354,9 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | in pipelines: 117(GETNEXT)
| | |
| | 116:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| | HDFS partitions=1824/1824 files=1824 size=200.95MB
+| | HDFS partitions=1824/1824 files=1824 size=200.96MB
| | stored statistics:
-| | table: rows=2.88M size=200.95MB
+| | table: rows=2.88M size=200.96MB
| | partitions: 1824/1824 rows=2.88M
| | columns: all
| | extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -3302,6 +3398,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | JOIN BUILD
| | | join-table-id=36 plan-id=37 cohort-id=01
| | | build expressions: tpcds_parquet.item.i_item_sk
+| | | runtime filters: RF073[min_max] <- tpcds_parquet.item.i_item_sk
| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | |
| | 205:EXCHANGE [BROADCAST]
@@ -3343,6 +3440,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | JOIN BUILD
| | | | join-table-id=37 plan-id=38 cohort-id=11
| | | | build expressions: i_brand_id, i_category_id, i_class_id
+| | | | runtime filters: RF081[min_max] <- i_brand_id, RF082[min_max] <- i_category_id, RF083[min_max] <- i_class_id
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | |
| | | 202:EXCHANGE [HASH(i_brand_id,i_class_id,i_category_id)]
@@ -3415,6 +3513,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | | JOIN BUILD
| | | | | join-table-id=39 plan-id=40 cohort-id=12
| | | | | build expressions: d3.d_date_sk
+| | | | | runtime filters: RF099[min_max] <- d3.d_date_sk
| | | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | | |
| | | | 198:EXCHANGE [BROADCAST]
@@ -3450,6 +3549,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | | JOIN BUILD
| | | | | join-table-id=40 plan-id=41 cohort-id=12
| | | | | build expressions: iws.i_item_sk
+| | | | | runtime filters: RF101[min_max] <- iws.i_item_sk
| | | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | | |
| | | | 197:EXCHANGE [BROADCAST]
@@ -3471,6 +3571,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | |
| | | 100:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| | | HDFS partitions=1/1 files=2 size=45.09MB
+| | | runtime filters: RF099[min_max] -> ws_sold_date_sk, RF101[min_max] -> ws_item_sk
| | | stored statistics:
| | | table: rows=719.38K size=45.09MB
| | | columns: all
@@ -3532,6 +3633,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | | JOIN BUILD
| | | | | join-table-id=42 plan-id=43 cohort-id=13
| | | | | build expressions: d2.d_date_sk
+| | | | | runtime filters: RF095[min_max] <- d2.d_date_sk
| | | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | | |
| | | | 193:EXCHANGE [BROADCAST]
@@ -3567,6 +3669,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | | JOIN BUILD
| | | | | join-table-id=43 plan-id=44 cohort-id=13
| | | | | build expressions: ics.i_item_sk
+| | | | | runtime filters: RF097[min_max] <- ics.i_item_sk
| | | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | | |
| | | | 192:EXCHANGE [BROADCAST]
@@ -3588,6 +3691,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | |
| | | 95:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| | | HDFS partitions=1/1 files=3 size=96.62MB
+| | | runtime filters: RF095[min_max] -> cs_sold_date_sk, RF097[min_max] -> cs_item_sk
| | | stored statistics:
| | | table: rows=1.44M size=96.62MB
| | | columns: all
@@ -3663,6 +3767,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | JOIN BUILD
| | | | join-table-id=45 plan-id=46 cohort-id=11
| | | | build expressions: iss.i_item_sk
+| | | | runtime filters: RF093[min_max] <- iss.i_item_sk
| | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | |
| | | 188:EXCHANGE [BROADCAST]
@@ -3674,6 +3779,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | Per-Instance Resources: mem-estimate=16.00MB mem-reservation=256.00KB thread-reservation=1
| | | 90:SCAN HDFS [tpcds_parquet.item iss, RANDOM]
| | | HDFS partitions=1/1 files=1 size=1.73MB
+| | | runtime filters: RF081[min_max] -> iss.i_brand_id, RF082[min_max] -> iss.i_category_id, RF083[min_max] -> iss.i_class_id
| | | stored statistics:
| | | table: rows=18.00K size=1.73MB
| | | columns: all
@@ -3683,9 +3789,10 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | in pipelines: 90(GETNEXT)
| | |
| | 89:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| | HDFS partitions=1824/1824 files=1824 size=200.95MB
+| | HDFS partitions=1824/1824 files=1824 size=200.96MB
+| | runtime filters: RF093[min_max] -> ss_item_sk
| | stored statistics:
-| | table: rows=2.88M size=200.95MB
+| | table: rows=2.88M size=200.96MB
| | partitions: 1824/1824 rows=2.88M
| | columns: all
| | extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -3706,6 +3813,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | JOIN BUILD
| | | join-table-id=46 plan-id=47 cohort-id=01
| | | build expressions: i_item_sk
+| | | runtime filters: RF075[min_max] <- i_item_sk
| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | |
| | 187:EXCHANGE [BROADCAST]
@@ -3717,6 +3825,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | Per-Instance Resources: mem-estimate=16.00MB mem-reservation=256.00KB thread-reservation=1
| | 86:SCAN HDFS [tpcds_parquet.item, RANDOM]
| | HDFS partitions=1/1 files=1 size=1.73MB
+| | runtime filters: RF073[min_max] -> tpcds_parquet.item.i_item_sk
| | stored statistics:
| | table: rows=18.00K size=1.73MB
| | columns: all
@@ -3738,7 +3847,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | JOIN BUILD
| | | join-table-id=47 plan-id=48 cohort-id=01
| | | build expressions: d_date_sk
-| | | runtime filters: RF088[bloom] <- d_date_sk
+| | | runtime filters: RF076[bloom] <- d_date_sk, RF077[min_max] <- d_date_sk
| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | |
| | 186:EXCHANGE [BROADCAST]
@@ -3763,7 +3872,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| |
| 85:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| HDFS partitions=1/1 files=2 size=45.09MB
-| runtime filters: RF088[bloom] -> ws_sold_date_sk
+| runtime filters: RF077[min_max] -> ws_sold_date_sk, RF073[min_max] -> ws_item_sk, RF075[min_max] -> ws_item_sk, RF076[bloom] -> ws_sold_date_sk
| stored statistics:
| table: rows=719.38K size=45.09MB
| columns: all
@@ -3829,6 +3938,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | | JOIN BUILD
| | | | | join-table-id=07 plan-id=08 cohort-id=03
| | | | | build expressions: d_date_sk
+| | | | | runtime filters: RF071[min_max] <- d_date_sk
| | | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | | |
| | | | 182:EXCHANGE [BROADCAST]
@@ -3853,6 +3963,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | |
| | | 80:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| | | HDFS partitions=1/1 files=2 size=45.09MB
+| | | runtime filters: RF071[min_max] -> ws_sold_date_sk
| | | stored statistics:
| | | table: rows=719.38K size=45.09MB
| | | columns: all
@@ -3874,6 +3985,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | | JOIN BUILD
| | | | | join-table-id=06 plan-id=07 cohort-id=03
| | | | | build expressions: d_date_sk
+| | | | | runtime filters: RF069[min_max] <- d_date_sk
| | | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | | |
| | | | 181:EXCHANGE [BROADCAST]
@@ -3898,6 +4010,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | |
| | | 77:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| | | HDFS partitions=1/1 files=3 size=96.62MB
+| | | runtime filters: RF069[min_max] -> cs_sold_date_sk
| | | stored statistics:
| | | table: rows=1.44M size=96.62MB
| | | columns: all
@@ -3942,9 +4055,9 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | in pipelines: 75(GETNEXT)
| | |
| | 74:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| | HDFS partitions=1824/1824 files=1824 size=200.95MB
+| | HDFS partitions=1824/1824 files=1824 size=200.96MB
| | stored statistics:
-| | table: rows=2.88M size=200.95MB
+| | table: rows=2.88M size=200.96MB
| | partitions: 1824/1824 rows=2.88M
| | columns: all
| | extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -3986,6 +4099,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | JOIN BUILD
| | | join-table-id=24 plan-id=25 cohort-id=01
| | | build expressions: tpcds_parquet.item.i_item_sk
+| | | runtime filters: RF037[min_max] <- tpcds_parquet.item.i_item_sk
| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | |
| | 177:EXCHANGE [BROADCAST]
@@ -4027,6 +4141,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | JOIN BUILD
| | | | join-table-id=25 plan-id=26 cohort-id=08
| | | | build expressions: i_brand_id, i_category_id, i_class_id
+| | | | runtime filters: RF045[min_max] <- i_brand_id, RF046[min_max] <- i_category_id, RF047[min_max] <- i_class_id
| | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | |
| | | 174:EXCHANGE [HASH(i_brand_id,i_class_id,i_category_id)]
@@ -4099,6 +4214,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | | JOIN BUILD
| | | | | join-table-id=27 plan-id=28 cohort-id=09
| | | | | build expressions: d3.d_date_sk
+| | | | | runtime filters: RF063[min_max] <- d3.d_date_sk
| | | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | | |
| | | | 170:EXCHANGE [BROADCAST]
@@ -4134,6 +4250,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | | JOIN BUILD
| | | | | join-table-id=28 plan-id=29 cohort-id=09
| | | | | build expressions: iws.i_item_sk
+| | | | | runtime filters: RF065[min_max] <- iws.i_item_sk
| | | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | | |
| | | | 169:EXCHANGE [BROADCAST]
@@ -4155,6 +4272,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | |
| | | 58:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| | | HDFS partitions=1/1 files=2 size=45.09MB
+| | | runtime filters: RF063[min_max] -> ws_sold_date_sk, RF065[min_max] -> ws_item_sk
| | | stored statistics:
| | | table: rows=719.38K size=45.09MB
| | | columns: all
@@ -4216,6 +4334,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | | JOIN BUILD
| | | | | join-table-id=30 plan-id=31 cohort-id=10
| | | | | build expressions: d2.d_date_sk
+| | | | | runtime filters: RF059[min_max] <- d2.d_date_sk
| | | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | | |
| | | | 165:EXCHANGE [BROADCAST]
@@ -4251,6 +4370,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | | JOIN BUILD
| | | | | join-table-id=31 plan-id=32 cohort-id=10
| | | | | build expressions: ics.i_item_sk
+| | | | | runtime filters: RF061[min_max] <- ics.i_item_sk
| | | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | | |
| | | | 164:EXCHANGE [BROADCAST]
@@ -4272,6 +4392,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | |
| | | 53:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| | | HDFS partitions=1/1 files=3 size=96.62MB
+| | | runtime filters: RF059[min_max] -> cs_sold_date_sk, RF061[min_max] -> cs_item_sk
| | | stored statistics:
| | | table: rows=1.44M size=96.62MB
| | | columns: all
@@ -4347,6 +4468,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | JOIN BUILD
| | | | join-table-id=33 plan-id=34 cohort-id=08
| | | | build expressions: iss.i_item_sk
+| | | | runtime filters: RF057[min_max] <- iss.i_item_sk
| | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | |
| | | 160:EXCHANGE [BROADCAST]
@@ -4358,6 +4480,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | Per-Instance Resources: mem-estimate=16.00MB mem-reservation=256.00KB thread-reservation=1
| | | 48:SCAN HDFS [tpcds_parquet.item iss, RANDOM]
| | | HDFS partitions=1/1 files=1 size=1.73MB
+| | | runtime filters: RF045[min_max] -> iss.i_brand_id, RF046[min_max] -> iss.i_category_id, RF047[min_max] -> iss.i_class_id
| | | stored statistics:
| | | table: rows=18.00K size=1.73MB
| | | columns: all
@@ -4367,9 +4490,10 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | in pipelines: 48(GETNEXT)
| | |
| | 47:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| | HDFS partitions=1824/1824 files=1824 size=200.95MB
+| | HDFS partitions=1824/1824 files=1824 size=200.96MB
+| | runtime filters: RF057[min_max] -> ss_item_sk
| | stored statistics:
-| | table: rows=2.88M size=200.95MB
+| | table: rows=2.88M size=200.96MB
| | partitions: 1824/1824 rows=2.88M
| | columns: all
| | extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -4390,6 +4514,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | JOIN BUILD
| | | join-table-id=34 plan-id=35 cohort-id=01
| | | build expressions: i_item_sk
+| | | runtime filters: RF039[min_max] <- i_item_sk
| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | |
| | 159:EXCHANGE [BROADCAST]
@@ -4401,6 +4526,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | Per-Instance Resources: mem-estimate=16.00MB mem-reservation=256.00KB thread-reservation=1
| | 44:SCAN HDFS [tpcds_parquet.item, RANDOM]
| | HDFS partitions=1/1 files=1 size=1.73MB
+| | runtime filters: RF037[min_max] -> tpcds_parquet.item.i_item_sk
| | stored statistics:
| | table: rows=18.00K size=1.73MB
| | columns: all
@@ -4422,7 +4548,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | JOIN BUILD
| | | join-table-id=35 plan-id=36 cohort-id=01
| | | build expressions: d_date_sk
-| | | runtime filters: RF046[bloom] <- d_date_sk
+| | | runtime filters: RF040[bloom] <- d_date_sk, RF041[min_max] <- d_date_sk
| | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | |
| | 158:EXCHANGE [BROADCAST]
@@ -4447,7 +4573,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| |
| 43:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| HDFS partitions=1/1 files=3 size=96.62MB
-| runtime filters: RF046[bloom] -> cs_sold_date_sk
+| runtime filters: RF041[min_max] -> cs_sold_date_sk, RF037[min_max] -> cs_item_sk, RF039[min_max] -> cs_item_sk, RF040[bloom] -> cs_sold_date_sk
| stored statistics:
| table: rows=1.44M size=96.62MB
| columns: all
@@ -4513,6 +4639,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | JOIN BUILD
| | | | join-table-id=03 plan-id=04 cohort-id=02
| | | | build expressions: d_date_sk
+| | | | runtime filters: RF035[min_max] <- d_date_sk
| | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | |
| | | 154:EXCHANGE [BROADCAST]
@@ -4537,6 +4664,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | |
| | 38:SCAN HDFS [tpcds_parquet.web_sales, RANDOM]
| | HDFS partitions=1/1 files=2 size=45.09MB
+| | runtime filters: RF035[min_max] -> ws_sold_date_sk
| | stored statistics:
| | table: rows=719.38K size=45.09MB
| | columns: all
@@ -4558,6 +4686,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | | JOIN BUILD
| | | | join-table-id=02 plan-id=03 cohort-id=02
| | | | build expressions: d_date_sk
+| | | | runtime filters: RF033[min_max] <- d_date_sk
| | | | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| | | |
| | | 153:EXCHANGE [BROADCAST]
@@ -4582,6 +4711,7 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | |
| | 35:SCAN HDFS [tpcds_parquet.catalog_sales, RANDOM]
| | HDFS partitions=1/1 files=3 size=96.62MB
+| | runtime filters: RF033[min_max] -> cs_sold_date_sk
| | stored statistics:
| | table: rows=1.44M size=96.62MB
| | columns: all
@@ -4626,9 +4756,9 @@ Per-Instance Resources: mem-estimate=76.80MB mem-reservation=43.88MB thread-rese
| | in pipelines: 33(GETNEXT)
| |
| 32:SCAN HDFS [tpcds_parquet.store_sales, RANDOM]
-| HDFS partitions=1824/1824 files=1824 size=200.95MB
+| HDFS partitions=1824/1824 files=1824 size=200.96MB
| stored statistics:
-| table: rows=2.88M size=200.95MB
+| table: rows=2.88M size=200.96MB
| partitions: 1824/1824 rows=2.88M
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=130.09K
@@ -4670,7 +4800,7 @@ Per-Instance Resources: mem-estimate=26.00MB mem-reservation=4.00MB thread-reser
| JOIN BUILD
| | join-table-id=12 plan-id=13 cohort-id=01
| | build expressions: tpcds_parquet.item.i_item_sk
-| | runtime filters: RF000[bloom] <- tpcds_parquet.item.i_item_sk
+| | runtime filters: RF000[bloom] <- tpcds_parquet.item.i_item_sk, RF001[min_max] <- tpcds_parquet.item.i_item_sk
| | mem-estimate=3.88MB mem-reservation=3.88MB spill-buffer=64.00KB thread-reservation=0
| |
| 149:EXCHANGE [BROADCAST]
... 30643 lines suppressed ...