You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "BiteTheDDDDt (via GitHub)" <gi...@apache.org> on 2023/07/24 07:55:27 UTC
[GitHub] [doris] BiteTheDDDDt opened a new pull request, #22152: [Bug](runtime-filter) fix probe expr prepare twice on minmax runtime filter
BiteTheDDDDt opened a new pull request, #22152:
URL: https://github.com/apache/doris/pull/22152
## Proposed changes
fix probe expr prepare twice on minmax runtime filter
```cpp
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_master/doris/be/src/common/signal_handler.h:413
1# 0x00007F8F52477090 in /lib/x86_64-linux-gnu/libc.so.6
2# raise at ../sysdeps/unix/sysv/linux/raise.c:51
3# abort at /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:81
4# 0x0000564C7F098FD9 in /mnt/ssd01/doris-master/VEC_UBSAN/be/lib/doris_be
5# 0x0000564C7F08E5ED in /mnt/ssd01/doris-master/VEC_UBSAN/be/lib/doris_be
6# google::LogMessage::SendToLog() in /mnt/ssd01/doris-master/VEC_UBSAN/be/lib/doris_be
7# google::LogMessage::Flush() in /mnt/ssd01/doris-master/VEC_UBSAN/be/lib/doris_be
8# google::LogMessageFatal::~LogMessageFatal() in /mnt/ssd01/doris-master/VEC_UBSAN/be/lib/doris_be
9# doris::vectorized::VExprContext::fn_context(int) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exprs/vexpr_context.h:64
10# doris::vectorized::VExpr::init_function_context(doris::vectorized::VExprContext*, doris::FunctionContext::FunctionStateScope, std::shared_ptr const&) const at /home/zcp/repo_center/doris_master/doris/be/src/vec/exprs/vexpr.cpp:410
11# doris::vectorized::VCastExpr::open(doris::RuntimeState*, doris::vectorized::VExprContext*, doris::FunctionContext::FunctionStateScope) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exprs/vcast_expr.cpp:85
12# doris::vectorized::VExpr::open(doris::RuntimeState*, doris::vectorized::VExprContext*, doris::FunctionContext::FunctionStateScope) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exprs/vexpr.cpp:115
13# doris::vectorized::VCaseExpr::open(doris::RuntimeState*, doris::vectorized::VExprContext*, doris::FunctionContext::FunctionStateScope) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exprs/vcase_expr.cpp:82
14# doris::vectorized::VExpr::open(doris::RuntimeState*, doris::vectorized::VExprContext*, doris::FunctionContext::FunctionStateScope) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exprs/vexpr.cpp:115
15# doris::vectorized::VectorizedFnCall::open(doris::RuntimeState*, doris::vectorized::VExprContext*, doris::FunctionContext::FunctionStateScope) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exprs/vectorized_fn_call.cpp:124
16# doris::vectorized::VRuntimeFilterWrapper::open(doris::RuntimeState*, doris::vectorized::VExprContext*, doris::FunctionContext::FunctionStateScope) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exprs/vruntimefilter_wrapper.cpp:66
17# doris::vectorized::VExprContext::open(doris::RuntimeState*) in /mnt/ssd01/doris-master/VEC_UBSAN/be/lib/doris_be
18# doris::vectorized::RuntimeFilterConsumer::_append_rf_into_conjuncts(std::vector, std::allocator > > const&) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exec/runtime_filter_consumer.cpp:124
19# doris::vectorized::RuntimeFilterConsumer::_acquire_runtime_filter() at /home/zcp/repo_center/doris_master/doris/be/src/vec/exec/runtime_filter_consumer.cpp:108
20# doris::vectorized::VScanNode::alloc_resource(doris::RuntimeState*) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exec/scan/vscan_node.cpp:171
21# doris::pipeline::StreamingOperator::open(doris::RuntimeState*) at /home/zcp/repo_center/doris_master/doris/be/src/pipeline/exec/operator.h:332
22# doris::pipeline::PipelineTask::_open() at /home/zcp/repo_center/doris_master/doris/be/src/pipeline/pipeline_task.cpp:179
23# doris::pipeline::PipelineTask::execute(bool*) at /home/zcp/repo_center/doris_master/doris/be/src/pipeline/pipeline_task.cpp:208
24# doris::pipeline::TaskScheduler::_do_work(unsigned long) at /home/zcp/repo_center/doris_master/doris/be/src/pipeline/task_scheduler.cpp:275
25# void std::__invoke_impl(std::__invoke_memfun_deref, void (doris::pipeline::TaskScheduler::*&)(unsigned long), doris::pipeline::TaskScheduler*&, unsigned long&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:74
26# std::__invoke_result::type std::__invoke(void (doris::pipeline::TaskScheduler::*&)(unsigned long), doris::pipeline::TaskScheduler*&, unsigned long&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96
27# void std::_Bind::__call(std::tuple<>&&, std::_Index_tuple<0ul, 1ul>) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:420
28# void std::_Bind::operator()<, void>() at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:503
29# void std::__invoke_impl&>(std::__invoke_other, std::_Bind&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
30# std::enable_if&>, void>::type std::__invoke_r&>(std::_Bind&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:117
31# std::_Function_handler >::_M_invoke(std::_Any_data const&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
32# std::function::operator()() const at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
33# doris::FunctionRunnable::run() at /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:48
34# doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:531
35# void std::__invoke_impl(std::__invoke_memfun_deref, void (doris::ThreadPool::*&)(), doris::ThreadPool*&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:74
36# std::__invoke_result::type std::__invoke(void (doris::ThreadPool::*&)(), doris::ThreadPool*&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96
37# void std::_Bind::__call(std::tuple<>&&, std::_Index_tuple<0ul>) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:420
38# void std::_Bind::operator()<, void>() at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:503
39# void std::__invoke_impl&>(std::__invoke_other, std::_Bind&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
40# std::enable_if&>, void>::type std::__invoke_r&>(std::_Bind&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:117
41# std::_Function_handler >::_M_invoke(std::_Any_data const&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
42# std::function::operator()() const at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
43# doris::Thread::supervise_thread(void*) at /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:465
44# start_thread at /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:478
45# __clone at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
172.21.0.22 last coredump sql:
```
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #22152: [Chore](refactor) remove some unused code
Posted by "BiteTheDDDDt (via GitHub)" <gi...@apache.org>.
BiteTheDDDDt commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1652897411
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #22152: [Chore](refactor) remove some unused code
Posted by "BiteTheDDDDt (via GitHub)" <gi...@apache.org>.
BiteTheDDDDt commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1654884817
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22152: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1647490786
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.24 seconds
stream load tsv: 507 seconds loaded 74807831229 Bytes, about 140 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162254269 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22152: [Chore](refactor) remove some unused code
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1652901122
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #22152: [Chore](refactor) remove some unused code
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #22152:
URL: https://github.com/apache/doris/pull/22152#discussion_r1275890764
##########
be/src/exprs/runtime_filter.h:
##########
@@ -368,8 +368,8 @@ class IRuntimeFilter {
// expr index
int _expr_order;
// used for await or signal
- doris::Mutex _inner_mutex;
Review Comment:
这种的别改，否则不知道是系统的还是我们的了
##########
be/src/exprs/runtime_filter.h:
##########
@@ -368,8 +368,8 @@ class IRuntimeFilter {
// expr index
int _expr_order;
// used for await or signal
- doris::Mutex _inner_mutex;
Review Comment:
这种的别改，否则不知道是系统的还是我们的了
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22152: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1648902252
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22152: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1647413483
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #22152: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
Posted by "BiteTheDDDDt (via GitHub)" <gi...@apache.org>.
BiteTheDDDDt commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1647401639
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22152: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1647744995
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22152: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1649001184
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.29 seconds
stream load tsv: 508 seconds loaded 74807831229 Bytes, about 140 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17168253644 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #22152: [Chore](refactor) remove some unused code
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #22152:
URL: https://github.com/apache/doris/pull/22152#discussion_r1275891134
##########
be/src/exprs/runtime_filter.cpp:
##########
@@ -1477,7 +1476,7 @@ void IRuntimeFilter::init_profile(RuntimeProfile* parent_profile) {
}
DCHECK(parent_profile != nullptr);
_name = fmt::format("RuntimeFilter: (id = {}, type = {})", _filter_id,
- ::doris::to_string(_runtime_filter_type));
Review Comment:
这种的不要动，能感受到是全局的，否则容易乱
##########
be/src/exprs/runtime_filter.cpp:
##########
@@ -1477,7 +1476,7 @@ void IRuntimeFilter::init_profile(RuntimeProfile* parent_profile) {
}
DCHECK(parent_profile != nullptr);
_name = fmt::format("RuntimeFilter: (id = {}, type = {})", _filter_id,
- ::doris::to_string(_runtime_filter_type));
Review Comment:
这种的不要动，能感受到是全局的，否则容易乱
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #22152: [Chore](refactor) remove some unused code
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #22152:
URL: https://github.com/apache/doris/pull/22152#discussion_r1275908493
##########
be/src/exprs/runtime_filter.cpp:
##########
@@ -1490,8 +1489,7 @@ void IRuntimeFilter::init_profile(RuntimeProfile* parent_profile) {
void IRuntimeFilter::update_runtime_filter_type_to_profile() {
if (_profile != nullptr) {
- _profile->add_info_string("RealRuntimeFilterType",
- ::doris::to_string(_wrapper->get_real_type()));
Review Comment:
and here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] HappenLee commented on a diff in pull request #22152: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on code in PR #22152:
URL: https://github.com/apache/doris/pull/22152#discussion_r1272665988
##########
be/src/exprs/runtime_filter.cpp:
##########
@@ -1899,8 +1894,20 @@ Status RuntimePredicateWrapper::get_push_exprs(std::vector<vectorized::VExprSPtr
cloned_expr = prob_expr->root()->clone();
min_pred->add_child(cloned_expr);
min_pred->add_child(min_literal);
- container->push_back(
- vectorized::VRuntimeFilterWrapper::create_shared(min_pred_node, min_pred));
+
+ TTypeDesc type_desc = create_type_desc(PrimitiveType::TYPE_BOOLEAN);
+ type_desc.__set_is_nullable(false);
+ TExprNode node;
+ node.__set_type(type_desc);
+ node.__set_node_type(TExprNodeType::COMPOUND_PRED);
+ node.__set_opcode(TExprOpcode::COMPOUND_AND);
+ node.__set_is_nullable(min_pred->is_nullable() || max_pred->is_nullable());
+
+ auto and_pred = vectorized::VCompoundPred::create_shared(node);
Review Comment:
change to and expr，maybe recheck the expr still can push down to storage layer？
##########
be/src/exprs/runtime_filter.cpp:
##########
@@ -1899,8 +1894,20 @@ Status RuntimePredicateWrapper::get_push_exprs(std::vector<vectorized::VExprSPtr
cloned_expr = prob_expr->root()->clone();
min_pred->add_child(cloned_expr);
min_pred->add_child(min_literal);
- container->push_back(
- vectorized::VRuntimeFilterWrapper::create_shared(min_pred_node, min_pred));
+
+ TTypeDesc type_desc = create_type_desc(PrimitiveType::TYPE_BOOLEAN);
+ type_desc.__set_is_nullable(false);
+ TExprNode node;
+ node.__set_type(type_desc);
+ node.__set_node_type(TExprNodeType::COMPOUND_PRED);
+ node.__set_opcode(TExprOpcode::COMPOUND_AND);
+ node.__set_is_nullable(min_pred->is_nullable() || max_pred->is_nullable());
+
+ auto and_pred = vectorized::VCompoundPred::create_shared(node);
Review Comment:
change to and expr，maybe recheck the expr still can push down to storage layer？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #22152: [Chore](refactor) remove some unused code
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #22152:
URL: https://github.com/apache/doris/pull/22152#discussion_r1275891676
##########
be/src/exprs/runtime_filter.cpp:
##########
@@ -1477,7 +1476,7 @@ void IRuntimeFilter::init_profile(RuntimeProfile* parent_profile) {
}
DCHECK(parent_profile != nullptr);
_name = fmt::format("RuntimeFilter: (id = {}, type = {})", _filter_id,
- ::doris::to_string(_runtime_filter_type));
Review Comment:
我们应该全局的函数调用，都加namespace 才对，否则区分不清楚是不是对象的函数
##########
be/src/exprs/runtime_filter.cpp:
##########
@@ -1477,7 +1476,7 @@ void IRuntimeFilter::init_profile(RuntimeProfile* parent_profile) {
}
DCHECK(parent_profile != nullptr);
_name = fmt::format("RuntimeFilter: (id = {}, type = {})", _filter_id,
- ::doris::to_string(_runtime_filter_type));
Review Comment:
我们应该全局的函数调用，都加namespace 才对，否则区分不清楚是不是对象的函数
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22152: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1648985882
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22152: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1647793571
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.34 seconds
stream load tsv: 537 seconds loaded 74807831229 Bytes, about 132 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.3 seconds inserted 10000000 Rows, about 341K ops/s
storage size: 17162302675 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #22152: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
Posted by "BiteTheDDDDt (via GitHub)" <gi...@apache.org>.
BiteTheDDDDt commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1647734682
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #22152: [Chore](refactor) remove some unused code
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #22152:
URL: https://github.com/apache/doris/pull/22152#discussion_r1275919837
##########
be/src/vec/exprs/vexpr.h:
##########
@@ -81,8 +81,6 @@ class VExpr {
VExpr() = default;
virtual ~VExpr() = default;
- virtual VExprSPtr clone() const = 0;
Review Comment:
This method is useless????
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22152: [Chore](refactor) remove some unused code
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1655369417
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] BiteTheDDDDt merged pull request #22152: [Chore](refactor) remove some unused code
Posted by "BiteTheDDDDt (via GitHub)" <gi...@apache.org>.
BiteTheDDDDt merged PR #22152:
URL: https://github.com/apache/doris/pull/22152
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] HappenLee commented on a diff in pull request #22152: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
Posted by "HappenLee (via GitHub)" <gi...@apache.org>.
HappenLee commented on code in PR #22152:
URL: https://github.com/apache/doris/pull/22152#discussion_r1272663886
##########
be/src/pipeline/pipeline_task.cpp:
##########
@@ -284,7 +284,7 @@ Status PipelineTask::finalize() {
}
Status PipelineTask::try_close() {
- _sink->try_close(_state);
+ RETURN_IF_ERROR(_sink->try_close(_state));
Review Comment:
sink try close error， still need to try close source
##########
be/src/pipeline/pipeline_task.cpp:
##########
@@ -284,7 +284,7 @@ Status PipelineTask::finalize() {
}
Status PipelineTask::try_close() {
- _sink->try_close(_state);
+ RETURN_IF_ERROR(_sink->try_close(_state));
Review Comment:
sink try close error， still need to try close source
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22152: [Chore](refactor) remove some unused code
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1652936426
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.9 seconds
stream load tsv: 507 seconds loaded 74807831229 Bytes, about 140 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17160306610 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22152: [Chore](refactor) remove some unused code
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1652897663
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #22152: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
Posted by "BiteTheDDDDt (via GitHub)" <gi...@apache.org>.
BiteTheDDDDt commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1648888999
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22152: [Chore](refactor) remove some unused code
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1654889366
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #22152: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
Posted by "BiteTheDDDDt (via GitHub)" <gi...@apache.org>.
BiteTheDDDDt commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1648974405
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22152: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1648902022
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22152: [Bug](runtime-filter) fix probe expr prepared twice on minmax runtime filter
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1649062225
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.92 seconds
stream load tsv: 505 seconds loaded 74807831229 Bytes, about 141 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17161275613 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #22152: [Chore](refactor) remove some unused code
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1654931586
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.39 seconds
stream load tsv: 506 seconds loaded 74807831229 Bytes, about 140 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.4 seconds inserted 10000000 Rows, about 340K ops/s
storage size: 17167290751 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #22152: [Chore](refactor) remove some unused code
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22152:
URL: https://github.com/apache/doris/pull/22152#issuecomment-1655369464
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org