You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@doris.apache.org by "xinyiZzz (via GitHub)" <gi...@apache.org> on 2023/04/10 01:32:25 UTC

[GitHub] [doris] xinyiZzz opened a new pull request, #18503: [enhancement](exception) Column filter/replicate supports exception safety

xinyiZzz opened a new pull request, #18503:
URL: https://github.com/apache/doris/pull/18503

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   column filter/replicate supports exception safety, which will actively cancel query when an exception is thrown.
   
   ## Checklist(Required)
   
   * [ ] Does it affect the original behavior
   * [ ] Has unit tests been added
   * [ ] Has document been added or modified
   * [ ] Does it need to update dependencies
   * [ ] Is this PR support rollback (If NO, please explain WHY)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] github-actions[bot] commented on a diff in pull request #18503: [enhancement](exception) Column filter/replicate supports exception safety

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] commented on code in PR #18503:
URL: https://github.com/apache/doris/pull/18503#discussion_r1161382440


##########
be/src/vec/columns/column_vector.cpp:
##########
@@ -523,9 +517,7 @@ ColumnPtr ColumnVector<T>::permute(const IColumn::Permutation& perm, size_t limi
 template <typename T>
 ColumnPtr ColumnVector<T>::replicate(const IColumn::Offsets& offsets) const {
     size_t size = data.size();
-    if (size != offsets.size()) {
-        LOG(FATAL) << "Size of offsets doesn't match size of column.";
-    }
+    column_match_offsets_size(size, offsets.size());

Review Comment:
   warning: use of undeclared identifier 'column_match_offsets_size' [clang-diagnostic-error]
   ```cpp
       column_match_offsets_size(size, offsets.size());
       ^
   ```
   



##########
be/src/vec/columns/columns_common.cpp:
##########
@@ -238,9 +236,7 @@
                                                     PaddedPODArray<OT>& offsets,
                                                     const IColumn::Filter& filter) {
     const size_t size = offsets.size();
-    if (offsets.size() != filter.size()) {
-        LOG(FATAL) << "Size of filter doesn't match size of column.";
-    }
+    column_match_filter_size(size, filter.size());

Review Comment:
   warning: use of undeclared identifier 'column_match_filter_size' [clang-diagnostic-error]
   ```cpp
       column_match_filter_size(size, filter.size());
       ^
   ```
   



##########
be/src/vec/exec/join/vhash_join_node.cpp:
##########
@@ -516,9 +516,7 @@ Status HashJoinNode::pull(doris::RuntimeState* state, vectorized::Block* output_
                     *_hash_table_variants, *_process_hashtable_ctx_variants,
                     make_bool_variant(_need_null_map_for_probe),
                     make_bool_variant(_probe_ignore_null));
-        } catch (const doris::Exception& e) {
-            return Status::Error(e.code(), e.to_string());
-        }
+        })

Review Comment:
   warning: expected ';' after do/while statement [clang-diagnostic-error]
   
   ```suggestion
           });
   ```
   



##########
be/src/vec/exec/varrow_scanner.cpp:
##########
@@ -251,7 +251,7 @@ Status VArrowScanner::get_next(vectorized::Block* block, bool* eof) {
     RETURN_IF_ERROR(_cast_src_block(&_src_block));
 
     // materialize, src block => dest columns
-    return _fill_dest_block(block, eof);
+    return RETURN_IF_CATCH_EXCEPTION(_fill_dest_block(block, eof));

Review Comment:
   warning: expected expression [clang-diagnostic-error]
   ```cpp
       return RETURN_IF_CATCH_EXCEPTION(_fill_dest_block(block, eof));
              ^
   ```
   **be/src/runtime/thread_context.h:315:** expanded from macro 'RETURN_IF_CATCH_EXCEPTION'
   ```cpp
       do {                                                                                       \
       ^
   ```
   



##########
be/src/vec/exec/join/vnested_loop_join_node.cpp:
##########
@@ -587,10 +589,11 @@
             const IColumn::Filter& filter =
                     assert_cast<const doris::vectorized::ColumnVector<UInt8>&>(*filter_column)
                             .get_data();
-            _do_filtering_and_update_visited_flags_impl<decltype(filter), SetBuildSideFlag,
-                                                        SetProbeSideFlag>(
-                    block, column_to_keep, build_block_idx, processed_blocks_num, materialize,
-                    filter);
+            RETURN_IF_CATCH_EXCEPTION(
+                    _do_filtering_and_update_visited_flags_impl<decltype(filter), SetBuildSideFlag,

Review Comment:
   warning: too many arguments provided to function-like macro invocation [clang-diagnostic-error]
   ```cpp
                       _do_filtering_and_update_visited_flags_impl<decltype(filter), SetBuildSideFlag,
                                                                                     ^
   ```
   **be/src/runtime/thread_context.h:314:** macro 'RETURN_IF_CATCH_EXCEPTION' defined here
   ```cpp
   #define RETURN_IF_CATCH_EXCEPTION(stmt)                                                        \
           ^
   ```
   



##########
be/src/vec/exprs/vexpr_context.cpp:
##########
@@ -38,12 +38,10 @@ VExprContext::~VExprContext() {
 
 doris::Status VExprContext::execute(doris::vectorized::Block* block, int* result_column_id) {
     Status st;
-    try {
+    RETURN_IF_CATCH_EXCEPTION({
         st = _root->execute(this, block, result_column_id);
         _last_result_column_id = *result_column_id;
-    } catch (const doris::Exception& e) {
-        st = Status::Error(e.code(), e.to_string());
-    }
+    })

Review Comment:
   warning: expected ';' after do/while statement [clang-diagnostic-error]
   
   ```suggestion
       });
   ```
   



##########
be/src/vec/core/block.h:
##########
@@ -132,7 +132,8 @@ class Block {
             raw_res_ptr = col_ptr_nullable->get_nested_column_ptr();
         }
 
-        return input_col_ptr->filter_by_selector(sel_rowid_idx, select_size, raw_res_ptr);
+        return RETURN_IF_CATCH_EXCEPTION(

Review Comment:
   warning: expected expression [clang-diagnostic-error]
   ```cpp
           return RETURN_IF_CATCH_EXCEPTION(
                  ^
   ```
   **be/src/runtime/thread_context.h:315:** expanded from macro 'RETURN_IF_CATCH_EXCEPTION'
   ```cpp
       do {                                                                                       \
       ^
   ```
   



##########
be/src/vec/exec/join/vnested_loop_join_node.cpp:
##########
@@ -587,10 +589,11 @@
             const IColumn::Filter& filter =
                     assert_cast<const doris::vectorized::ColumnVector<UInt8>&>(*filter_column)
                             .get_data();
-            _do_filtering_and_update_visited_flags_impl<decltype(filter), SetBuildSideFlag,
-                                                        SetProbeSideFlag>(
-                    block, column_to_keep, build_block_idx, processed_blocks_num, materialize,
-                    filter);
+            RETURN_IF_CATCH_EXCEPTION(

Review Comment:
   warning: use of undeclared identifier 'RETURN_IF_CATCH_EXCEPTION' [clang-diagnostic-error]
   ```cpp
               RETURN_IF_CATCH_EXCEPTION(
               ^
   ```
   



##########
be/src/vec/exec/join/vnested_loop_join_node.cpp:
##########
@@ -555,10 +556,11 @@
                     filter_data[i] &= !null_map[i];
                 }
             }
-            _do_filtering_and_update_visited_flags_impl<decltype(filter), SetBuildSideFlag,
-                                                        SetProbeSideFlag>(
-                    block, column_to_keep, build_block_idx, processed_blocks_num, materialize,
-                    filter);
+            RETURN_IF_CATCH_EXCEPTION(
+                    _do_filtering_and_update_visited_flags_impl<decltype(filter), SetBuildSideFlag,

Review Comment:
   warning: too many arguments provided to function-like macro invocation [clang-diagnostic-error]
   ```cpp
                       _do_filtering_and_update_visited_flags_impl<decltype(filter), SetBuildSideFlag,
                                                                                     ^
   ```
   **be/src/runtime/thread_context.h:314:** macro 'RETURN_IF_CATCH_EXCEPTION' defined here
   ```cpp
   #define RETURN_IF_CATCH_EXCEPTION(stmt)                                                        \
           ^
   ```
   



##########
be/src/vec/columns/columns_common.cpp:
##########
@@ -158,9 +158,7 @@ void filter_arrays_impl_generic(const PaddedPODArray<T>& src_elems,
                                 PaddedPODArray<OT>* res_offsets, const IColumn::Filter& filt,
                                 ssize_t result_size_hint) {
     const size_t size = src_offsets.size();
-    if (size != filt.size()) {
-        LOG(FATAL) << "Size of filter doesn't match size of column.";
-    }
+    column_match_filter_size(size, filt.size());

Review Comment:
   warning: use of undeclared identifier 'column_match_filter_size' [clang-diagnostic-error]
   ```cpp
       column_match_filter_size(size, filt.size());
       ^
   ```
   



##########
be/src/vec/exec/join/vnested_loop_join_node.cpp:
##########
@@ -555,10 +556,11 @@ Status VNestedLoopJoinNode::_do_filtering_and_update_visited_flags(Block* block,
                     filter_data[i] &= !null_map[i];
                 }
             }
-            _do_filtering_and_update_visited_flags_impl<decltype(filter), SetBuildSideFlag,
-                                                        SetProbeSideFlag>(
-                    block, column_to_keep, build_block_idx, processed_blocks_num, materialize,
-                    filter);
+            RETURN_IF_CATCH_EXCEPTION(

Review Comment:
   warning: use of undeclared identifier 'RETURN_IF_CATCH_EXCEPTION' [clang-diagnostic-error]
   ```cpp
               RETURN_IF_CATCH_EXCEPTION(
               ^
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] xinyiZzz commented on a diff in pull request #18503: [enhancement](exception) Column filter/replicate supports exception safety

Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.

xinyiZzz commented on code in PR #18503:
URL: https://github.com/apache/doris/pull/18503#discussion_r1162768703


##########
be/src/runtime/thread_context.h:
##########
@@ -312,33 +312,22 @@ class StopCheckThreadMemTrackerLimit {
     tracker->transfer_to(                               \
             size, doris::thread_context()->thread_mem_tracker_mgr->limiter_mem_tracker_raw())
 
-// Consider catching other memory errors, such as memset failure, etc.
-#define RETURN_IF_CATCH_BAD_ALLOC(stmt)                                                            \
-    do {                                                                                           \
-        doris::thread_context()->thread_mem_tracker_mgr->clear_exceed_mem_limit_msg();             \
-        if (doris::enable_thread_catch_bad_alloc) {                                                \
-            try {                                                                                  \
-                { stmt; }                                                                          \
-            } catch (std::bad_alloc const& e) {                                                    \
-                doris::thread_context()->thread_mem_tracker()->print_log_usage(                    \
-                        doris::thread_context()->thread_mem_tracker_mgr->exceed_mem_limit_msg());  \
-                return Status::MemoryLimitExceeded(fmt::format(                                    \
-                        "PreCatch {}, {}", e.what(),                                               \
-                        doris::thread_context()->thread_mem_tracker_mgr->exceed_mem_limit_msg())); \
-            }                                                                                      \
-        } else {                                                                                   \
-            try {                                                                                  \
-                doris::enable_thread_catch_bad_alloc = true;                                       \
-                Defer defer {[&]() { doris::enable_thread_catch_bad_alloc = false; }};             \
-                { stmt; }                                                                          \
-            } catch (std::bad_alloc const& e) {                                                    \
-                doris::thread_context()->thread_mem_tracker()->print_log_usage(                    \
-                        doris::thread_context()->thread_mem_tracker_mgr->exceed_mem_limit_msg());  \
-                return Status::MemoryLimitExceeded(fmt::format(                                    \
-                        "PreCatch {}, {}", e.what(),                                               \
-                        doris::thread_context()->thread_mem_tracker_mgr->exceed_mem_limit_msg())); \
-            }                                                                                      \
-        }                                                                                          \
+#define RETURN_IF_CATCH_EXCEPTION(stmt)                                                        \
+    do {                                                                                       \
+        try {                                                                                  \
+            doris::thread_context()->thread_mem_tracker_mgr->clear_exceed_mem_limit_msg();     \
+            enable_thread_catch_bad_alloc++;                                                   \
+            { stmt; }                                                                          \
+            enable_thread_catch_bad_alloc--;                                                   \

Review Comment:
   fixed



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org