(datafusion) branch main updated: fix: Skip buffered rows which are not joined with streamed side when checking join filter results (#12159)

[vi...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

viirya pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion.git


The following commit(s) were added to refs/heads/main by this push:
     new dff590bfd2 fix: Skip buffered rows which are not joined with streamed side when checking join filter results (#12159)
dff590bfd2 is described below

commit dff590bfd2bb9993b2c8ce6f76a3bdd973e520a8
Author: Liang-Chi Hsieh <vi...@gmail.com>
AuthorDate: Mon Aug 26 13:57:17 2024 -0700

    fix: Skip buffered rows which are not joined with streamed side when checking join filter results (#12159)
---
 datafusion/physical-plan/src/joins/sort_merge_join.rs | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/datafusion/physical-plan/src/joins/sort_merge_join.rs b/datafusion/physical-plan/src/joins/sort_merge_join.rs
index 56cd699bf6..511cb4c55f 100644
--- a/datafusion/physical-plan/src/joins/sort_merge_join.rs
+++ b/datafusion/physical-plan/src/joins/sort_merge_join.rs
@@ -1474,6 +1474,12 @@ impl SMJStream {
                                 [chunk.buffered_batch_idx.unwrap()];
 
                             for i in 0..pre_mask.len() {
+                                // If the buffered row is not joined with streamed side,
+                                // skip it.
+                                if buffered_indices.is_null(i) {
+                                    continue;
+                                }
+
                                 let buffered_index = buffered_indices.value(i);
 
                                 buffered_batch.join_filter_failed_map.insert(


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org