[GitHub] [doris] Gabriel39 commented on a diff in pull request #16402: [fix](hashjoin) join produce blocks with rows larger than batch size: handle join with other conjuncts

["Gabriel39 (via GitHub)" <gi...@apache.org>]

Gabriel39 commented on code in PR #16402:
URL: https://github.com/apache/doris/pull/16402#discussion_r1099672507


##########
be/src/vec/exec/join/process_hash_table_probe_impl.h:
##########
@@ -462,95 +463,162 @@ Status ProcessHashTableProbe<JoinOpType>::do_process_with_other_join_conjuncts(
 
         bool all_match_one = true;
         int last_probe_index = probe_index;
-        while (probe_index < probe_rows) {
-            // ignore null rows
-            if constexpr (ignore_null && need_null_map_for_probe) {
-                if ((*null_map)[probe_index]) {
-                    if constexpr (probe_all) {
-                        _items_counts[probe_index++] = (uint32_t)1;
-                        same_to_prev.emplace_back(false);
-                        visited_map.emplace_back(nullptr);
-                        // only full outer / left outer need insert the data of right table
-                        if (LIKELY(current_offset < _build_block_rows.size())) {
-                            _build_block_offsets[current_offset] = -1;
-                            _build_block_rows[current_offset] = -1;
-                        } else {
-                            _build_block_offsets.emplace_back(-1);
-                            _build_block_rows.emplace_back(-1);
-                        }
-                        ++current_offset;
-                    } else {
-                        _items_counts[probe_index++] = (uint32_t)0;
-                    }
-                    all_match_one = false;
-                    continue;
+
+        int row_count_from_last_probe = 0;
+        bool is_the_last_sub_block = false;
+        size_t probe_size = 0;
+        auto& probe_row_match_iter =
+                std::get<ForwardIterator<Mapped>>(_join_node->_probe_row_match_iter);
+        if (probe_row_match_iter.ok()) {
+            auto origin_offset = current_offset;
+            for (; probe_row_match_iter.ok() && current_offset < _batch_size;
+                 ++probe_row_match_iter) {
+                if (LIKELY(current_offset < _build_block_rows.size())) {
+                    _build_block_offsets[current_offset] = probe_row_match_iter->block_offset;
+                    _build_block_rows[current_offset] = probe_row_match_iter->row_num;
+                } else {
+                    _build_block_offsets.emplace_back(probe_row_match_iter->block_offset);
+                    _build_block_rows.emplace_back(probe_row_match_iter->row_num);
                 }
+                ++current_offset;
+                visited_map.emplace_back(&probe_row_match_iter->visited);
+            }
+            same_to_prev.emplace_back(false);

Review Comment:
   is this always false needed to be emplace_back?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org