You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by mo...@apache.org on 2023/04/27 15:33:11 UTC
[doris] branch branch-2.0-alpha updated (fd2f8a6baa -> 8b7869e036)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch branch-2.0-alpha
in repository https://gitbox.apache.org/repos/asf/doris.git
from fd2f8a6baa [fix](memory) Fix AggFunc memory leak due to incorrect destroy (#19126)
new f3cbc48d9e [Chore](asan) set decrementOutputRecursionDepth to suppressions and remove some unu… (#18845)
new 1920da6001 [bugfix](memoryleak) close expr after it is pushdown to storage layer (#18849) (#18852)
new 8b7869e036 [fix](fe)hll_raw_agg function's return type should be always not nullable (#19115)
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.licenserc.yaml | 1 -
.rat-excludes | 1 -
be/CMakeLists.txt | 1 +
be/src/vec/exec/scan/vscan_node.cpp | 10 +++++++++
bin/start_be.sh | 5 +++--
conf/asan_suppr.conf | 3 +--
conf/be.conf | 2 +-
conf/fe.conf | 2 +-
conf/{asan_suppr.conf => lsan_suppr.conf} | 2 +-
.../apache/doris/catalog/AggregateFunction.java | 2 +-
tsan_suppressions | 24 ----------------------
11 files changed, 19 insertions(+), 34 deletions(-)
copy conf/{asan_suppr.conf => lsan_suppr.conf} (98%)
delete mode 100644 tsan_suppressions
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[doris] 02/03: [bugfix](memoryleak) close expr after it is pushdown to storage layer (#18849) (#18852)
Posted by mo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch branch-2.0-alpha
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 1920da60013bd7ec45a45a8ee1762a88592d577a
Author: yiguolei <67...@qq.com>
AuthorDate: Fri Apr 21 05:21:16 2023 +0800
[bugfix](memoryleak) close expr after it is pushdown to storage layer (#18849) (#18852)
Co-authored-by: yiguolei <yi...@gmail.com>
---
be/src/vec/exec/scan/vscan_node.cpp | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/be/src/vec/exec/scan/vscan_node.cpp b/be/src/vec/exec/scan/vscan_node.cpp
index 22378d2744..553a611076 100644
--- a/be/src/vec/exec/scan/vscan_node.cpp
+++ b/be/src/vec/exec/scan/vscan_node.cpp
@@ -638,6 +638,16 @@ Status VScanNode::_normalize_predicate(VExpr* conjunct_expr_root, VExpr** output
*output_expr = conjunct_expr_root;
return Status::OK();
} else {
+ if (left_child == nullptr) {
+ conjunct_expr_root->children()[0]->close(
+ _state, *_vconjunct_ctx_ptr,
+ (*_vconjunct_ctx_ptr)->get_function_state_scope());
+ }
+ if (right_child == nullptr) {
+ conjunct_expr_root->children()[1]->close(
+ _state, *_vconjunct_ctx_ptr,
+ (*_vconjunct_ctx_ptr)->get_function_state_scope());
+ }
// here only close the and expr self, do not close the child
conjunct_expr_root->set_children({});
conjunct_expr_root->close(_state, *_vconjunct_ctx_ptr,
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[doris] 01/03: [Chore](asan) set decrementOutputRecursionDepth to suppressions and remove some unu… (#18845)
Posted by mo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch branch-2.0-alpha
in repository https://gitbox.apache.org/repos/asf/doris.git
commit f3cbc48d9e7f1e0a6bf4ecd56e8218a926bd073e
Author: Pxl <px...@qq.com>
AuthorDate: Thu Apr 20 23:33:25 2023 +0800
[Chore](asan) set decrementOutputRecursionDepth to suppressions and remove some unu… (#18845)
18845
---
.licenserc.yaml | 1 -
.rat-excludes | 1 -
be/CMakeLists.txt | 1 +
bin/start_be.sh | 5 +++--
conf/asan_suppr.conf | 3 +--
conf/be.conf | 2 +-
conf/fe.conf | 2 +-
conf/{asan_suppr.conf => lsan_suppr.conf} | 2 +-
tsan_suppressions | 24 ------------------------
9 files changed, 8 insertions(+), 33 deletions(-)
diff --git a/.licenserc.yaml b/.licenserc.yaml
index 4d7443862a..98886a29ef 100644
--- a/.licenserc.yaml
+++ b/.licenserc.yaml
@@ -37,7 +37,6 @@ header:
- "**/*.sql"
- "**/*.lock"
- "**/*.out"
- - "tsan_suppressions"
- "docs/.markdownlintignore"
- "fe/fe-core/src/test/resources/data/net_snmp_normal"
- "fe/fe-core/src/main/antlr4/org/apache/doris/nereids/JavaLexer.g4"
diff --git a/.rat-excludes b/.rat-excludes
index 0a2f42e9e9..bb97aca3dc 100644
--- a/.rat-excludes
+++ b/.rat-excludes
@@ -35,5 +35,4 @@ condition_variable.*
murmur_hash3.*
utf8_check.cpp
.markdownlintignore
-tsan_suppressions
sse2neon.h
diff --git a/be/CMakeLists.txt b/be/CMakeLists.txt
index 3b767b410b..10e6fd09e6 100644
--- a/be/CMakeLists.txt
+++ b/be/CMakeLists.txt
@@ -1016,6 +1016,7 @@ install(FILES
${BASE_DIR}/../conf/be.conf
${BASE_DIR}/../conf/odbcinst.ini
${BASE_DIR}/../conf/asan_suppr.conf
+ ${BASE_DIR}/../conf/lsan_suppr.conf
${BASE_DIR}/../conf/hdfs-site.xml
DESTINATION ${OUTPUT_DIR}/conf)
diff --git a/bin/start_be.sh b/bin/start_be.sh
index bbaea90c02..f6e564149c 100755
--- a/bin/start_be.sh
+++ b/bin/start_be.sh
@@ -144,8 +144,9 @@ export ODBCSYSINI="${DORIS_HOME}/conf"
# support utf8 for oracle database
export NLS_LANG='AMERICAN_AMERICA.AL32UTF8'
-#filter known leak for lsan.
-export LSAN_OPTIONS="suppressions=${DORIS_HOME}/conf/asan_suppr.conf"
+# filter known leak.
+export LSAN_OPTIONS="suppressions=${DORIS_HOME}/conf/lsan_suppr.conf"
+export ASAN_OPTIONS="suppressions=${DORIS_HOME}/conf/asan_suppr.conf"
while read -r line; do
envline="$(echo "${line}" |
diff --git a/conf/asan_suppr.conf b/conf/asan_suppr.conf
index 254e6901ac..3c43b92e44 100644
--- a/conf/asan_suppr.conf
+++ b/conf/asan_suppr.conf
@@ -15,5 +15,4 @@
# specific language governing permissions and limitations
# under the License.
-# This is a known leak.
-leak:brpc
\ No newline at end of file
+interceptor_via_lib:apache::thrift
diff --git a/conf/be.conf b/conf/be.conf
index 1961ab3592..61c399b1e1 100644
--- a/conf/be.conf
+++ b/conf/be.conf
@@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.
-CUR_DATE='$(date +%Y%m%d-%H%M%S)'
+CUR_DATE=`date +%Y%m%d-%H%M%S`
PPROF_TMPDIR="$DORIS_HOME/log/"
diff --git a/conf/fe.conf b/conf/fe.conf
index 849d387bdd..77e2b6efa5 100644
--- a/conf/fe.conf
+++ b/conf/fe.conf
@@ -21,7 +21,7 @@
## see fe/src/org/apache/doris/common/Config.java
#####################################################################
-CUR_DATE='$(date +%Y%m%d-%H%M%S)'
+CUR_DATE=`date +%Y%m%d-%H%M%S`
# the output dir of stderr and stdout
LOG_DIR = ${DORIS_HOME}/log
diff --git a/conf/asan_suppr.conf b/conf/lsan_suppr.conf
similarity index 98%
copy from conf/asan_suppr.conf
copy to conf/lsan_suppr.conf
index 254e6901ac..fe294d58e8 100644
--- a/conf/asan_suppr.conf
+++ b/conf/lsan_suppr.conf
@@ -16,4 +16,4 @@
# under the License.
# This is a known leak.
-leak:brpc
\ No newline at end of file
+leak:brpc
diff --git a/tsan_suppressions b/tsan_suppressions
deleted file mode 100644
index a01346fe74..0000000000
--- a/tsan_suppressions
+++ /dev/null
@@ -1,24 +0,0 @@
-mutex:boost::condition_variable::wait(boost::unique_lock<boost::mutex>&)
-mutex:brpc::*
-mutex:doris::ConditionVariable::wait_until(doris::MonoTime const&) const
-mutex:doris::ConditionVariable::wait() const
-race:boost::intrusive::list_node_traits<void*>::get_next(boost::intrusive::list_node<void*> const* const&)
-race:brpc::*
-race:butil::*
-race:bvar::*
-race:doris::CountDownLatch::wait_until(doris::MonoTime const&) const
-race:doris::PBackendService::*
-race:doris::PStatus::status_code() const
-race:doris::PTabletWriterAddBatchResult::*
-race:doris::PTabletWriterOpenResult::*
-race:doris::RefCountClosure<doris::PTabletWriterOpenResult>::unref()
-race:doris::stream_load::TestInternalService::tablet_writer_add_batch(google::protobuf::RpcController*, doris::PTabletWriterAddBatchRequest const*, doris::PTabletWriterAddBatchResult*, google::protobuf::Closure*)
-race:glog_internal_namespace_::*
-race:google::protobuf::*
-race:operator delete(void*)
-race:std::_Bit_reference::operator bool() const
-race:std::char_traits<char>::compare(char const*, char const*, unsigned long)
-race:std::char_traits<char>::copy(char*, char const*, unsigned long)
-race:std::lock_guard<int volatile>::lock_guard(int volatile&)
-race:std::lock_guard<int volatile>::~lock_guard()
-race:void google::protobuf::internal::RepeatedPtrFieldBase::Clear<google::protobuf::RepeatedPtrField<doris::PTabletInfo>::TypeHandler>()
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[doris] 03/03: [fix](fe)hll_raw_agg function's return type should be always not nullable (#19115)
Posted by mo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch branch-2.0-alpha
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 8b7869e036dc25569af278250c6708527c20a626
Author: starocean999 <40...@users.noreply.github.com>
AuthorDate: Thu Apr 27 08:00:49 2023 +0800
[fix](fe)hll_raw_agg function's return type should be always not nullable (#19115)
---
.../src/main/java/org/apache/doris/catalog/AggregateFunction.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/AggregateFunction.java b/fe/fe-core/src/main/java/org/apache/doris/catalog/AggregateFunction.java
index 324b043541..c3ae7709ed 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/AggregateFunction.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/AggregateFunction.java
@@ -48,7 +48,7 @@ public class AggregateFunction extends Function {
public static ImmutableSet<String> NOT_NULLABLE_AGGREGATE_FUNCTION_NAME_SET = ImmutableSet.of("row_number", "rank",
"dense_rank", "multi_distinct_count", "multi_distinct_sum", FunctionSet.HLL_UNION_AGG,
- FunctionSet.HLL_UNION, FunctionSet.BITMAP_UNION, FunctionSet.BITMAP_INTERSECT,
+ FunctionSet.HLL_UNION, FunctionSet.HLL_RAW_AGG, FunctionSet.BITMAP_UNION, FunctionSet.BITMAP_INTERSECT,
FunctionSet.ORTHOGONAL_BITMAP_INTERSECT, FunctionSet.ORTHOGONAL_BITMAP_INTERSECT_COUNT,
FunctionSet.ORTHOGONAL_BITMAP_EXPR_CALCULATE_COUNT, FunctionSet.ORTHOGONAL_BITMAP_EXPR_CALCULATE,
FunctionSet.INTERSECT_COUNT, FunctionSet.ORTHOGONAL_BITMAP_UNION_COUNT,
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org