You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Zihao Ye (Code Review)" <ge...@cloudera.org> on 2024/01/12 10:51:55 UTC

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Zihao Ye has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20891


Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................

IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join

IMPALA-12582 reported an issue where enabling MIN_MAX RuntimeFilter for
a specific query would cause the executor to crash. The direct cause of
the crash was an out-of-bounds access to input_vals in the
ScalarFnCall::InterpretEval() function, but the root cause was actually
due to the related ScalarExprEvaluator not invoking the Open() function.
due to the related ScalarExprEvaluator not invoking Open(). This patch
fix this issue and adds corresponding test cases in
test_join_queries.py.

Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
---
M be/src/exec/nested-loop-join-builder.cc
M tests/query_test/test_join_queries.py
2 files changed, 21 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/20891/1
-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 1
Gerrit-Owner: Zihao Ye <ey...@163.com>

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/14959/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 3
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 09:40:42 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Zihao Ye (Code Review)" <ge...@cloudera.org>.
Hello Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/20891

to look at the new patch set (#4).

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................

IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join

This patch fixes an issue reported by IMPALA-12582, where enabling
MIN_MAX RuntimeFilter for a specific query would cause the executor to
crash. The direct cause of the crash was an out-of-bounds access to
input_vals in the ScalarFnCall::InterpretEval() function, but the root
cause was actually due to the related ScalarExprEvaluator not invoking
the Open() function.

Testing:
 - Added new E2E test case about this issue.

Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
---
M be/src/exec/nested-loop-join-builder.cc
M testdata/workloads/functional-query/queries/QueryTest/overlap_min_max_filters_on_partition_columns.test
2 files changed, 19 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/20891/4
-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 4
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Zihao Ye (Code Review)" <ge...@cloudera.org>.
Zihao Ye has posted comments on this change. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/20891/3/tests/query_test/test_runtime_filters.py
File tests/query_test/test_runtime_filters.py:

http://gerrit.cloudera.org:8080/#/c/20891/3/tests/query_test/test_runtime_filters.py@337
PS3, Line 337:     add_exec_option_dimension(cls, 'disable_codegen', [False, True])
> I think that this is not really necessary, as in exhaustive builds the test
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 3
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 11:36:55 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/14946/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 2
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 12 Jan 2024 12:08:21 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/20891/3/tests/query_test/test_runtime_filters.py
File tests/query_test/test_runtime_filters.py:

http://gerrit.cloudera.org:8080/#/c/20891/3/tests/query_test/test_runtime_filters.py@337
PS3, Line 337:     add_exec_option_dimension(cls, 'disable_codegen', [False, True])
I think that this is not really necessary, as in exhaustive builds the tests will be executed in both ways. So while this issue could pass precommit tests, periodic exhaustive tests would catch it.
see https://github.com/apache/impala/blob/e071e6f053ceb8cb4536837f76e76610a34aa8f2/tests/common/impala_test_suite.py#L1112

It would be good to run most tests both with and without codegen in core tests, but this would make the test runs significantly longer.



-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 3
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 11:17:31 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Patch Set 4:

Thanks for fixing this!


-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 4
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 12:00:47 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Zihao Ye (Code Review)" <ge...@cloudera.org>.
Zihao Ye has posted comments on this change. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Patch Set 2:

(2 comments)

Thank you for the code review!

http://gerrit.cloudera.org:8080/#/c/20891/2/tests/query_test/test_join_queries.py
File tests/query_test/test_join_queries.py:

http://gerrit.cloudera.org:8080/#/c/20891/2/tests/query_test/test_join_queries.py@116
PS2, Line 116:   def test_nested_loop_joins_with_min_max_runtime_filter(self, vector, unique_database):
> Checked the original patch that added the bug and it did add a test with ne
Yes, I simplified the triggering query. In the simplest case, it only needs two conditions to trigger, as follows:

set disable_codegen=true;
select col from tbl where col <= (select abs(max(col2)) from tbl2);

First, codegen needs to be disabled, and the column 'col' in the query should be a sorted or partitioned column so that we can apply min-max runtime filter. Additionally, the outer layer of the subsequent aggregate function (in this case, max) requires a function call expression (in this case, abs), which of course can be another suitable function. The function call expression can also be a more complex nested call. In some complexed expressions, set disable_codegen=true is also not necessary to trigger the issue, just like the original triggering query. Maybe due to the codegen does not cover all code paths, I guess.
As for the old query, it mainly lacks the outer layer of a function call expression for the aggregate function, hence it does not trigger the issue.


http://gerrit.cloudera.org:8080/#/c/20891/2/tests/query_test/test_join_queries.py@120
PS2, Line 120:         "CREATE TABLE {0} (id int) PARTITIONED BY(dt string) "
> optional: I think that this would be more readable if it was moved to an ex
Done, moving it to overlap_min_max_filters_on_partition_columns.test would be more appropriate.



-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 2
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 09:47:00 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Patch Set 5: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/10157/


-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 5
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 17:54:11 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has removed a vote on this change.

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Removed Verified-1 by Impala Public Jenkins <im...@cloudera.com>
-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 5
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Patch Set 5: Verified+1 Code-Review+2

The failed test seems unrelated, I am merging the patch


-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 5
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 18:40:22 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Zihao Ye (Code Review)" <ge...@cloudera.org>.
Zihao Ye has posted comments on this change. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/20891/4/testdata/workloads/functional-query/queries/QueryTest/overlap_min_max_filters_on_sorted_columns.test
File testdata/workloads/functional-query/queries/QueryTest/overlap_min_max_filters_on_sorted_columns.test:

http://gerrit.cloudera.org:8080/#/c/20891/4/testdata/workloads/functional-query/queries/QueryTest/overlap_min_max_filters_on_sorted_columns.test@362
PS4, Line 362: where l_orderkey < (select min(l_partkey) from
> Was removing abs intentional?
Yes, I think the test cases added in overlap_min_max_filters_on_partition_columns are sufficient to cover the issue. Adding "abs" here may not make much sense. Do you think it's necessary?



-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 4
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 11:52:17 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/20891/1/tests/query_test/test_join_queries.py
File tests/query_test/test_join_queries.py:

http://gerrit.cloudera.org:8080/#/c/20891/1/tests/query_test/test_join_queries.py@121
PS1, Line 121: t
flake8: E122 continuation line missing indentation or outdented



-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 1
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 12 Jan 2024 10:53:01 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/14960/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 4
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 12:02:41 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10157/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 5
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 12:01:16 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/14945/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 1
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 12 Jan 2024 11:18:38 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Patch Set 4: Code-Review+2

(1 comment)

http://gerrit.cloudera.org:8080/#/c/20891/4/testdata/workloads/functional-query/queries/QueryTest/overlap_min_max_filters_on_sorted_columns.test
File testdata/workloads/functional-query/queries/QueryTest/overlap_min_max_filters_on_sorted_columns.test:

http://gerrit.cloudera.org:8080/#/c/20891/4/testdata/workloads/functional-query/queries/QueryTest/overlap_min_max_filters_on_sorted_columns.test@362
PS4, Line 362: where l_orderkey < (select min(l_partkey) from
> Yes, I think the test cases added in overlap_min_max_filters_on_partition_c
No, it's ok like this.



-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 4
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 12:00:16 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Patch Set 5: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 5
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 12:01:15 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................

IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join

This patch fixes an issue reported by IMPALA-12582, where enabling
MIN_MAX RuntimeFilter for a specific query would cause the executor to
crash. The direct cause of the crash was an out-of-bounds access to
input_vals in the ScalarFnCall::InterpretEval() function, but the root
cause was actually due to the related ScalarExprEvaluator not invoking
the Open() function.

Testing:
 - Added new E2E test case about this issue.

Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Reviewed-on: http://gerrit.cloudera.org:8080/20891
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Csaba Ringhofer <cs...@cloudera.com>
Tested-by: Csaba Ringhofer <cs...@cloudera.com>
---
M be/src/exec/nested-loop-join-builder.cc
M testdata/workloads/functional-query/queries/QueryTest/overlap_min_max_filters_on_partition_columns.test
2 files changed, 19 insertions(+), 0 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved
  Csaba Ringhofer: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 6
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/20891/4/testdata/workloads/functional-query/queries/QueryTest/overlap_min_max_filters_on_sorted_columns.test
File testdata/workloads/functional-query/queries/QueryTest/overlap_min_max_filters_on_sorted_columns.test:

http://gerrit.cloudera.org:8080/#/c/20891/4/testdata/workloads/functional-query/queries/QueryTest/overlap_min_max_filters_on_sorted_columns.test@362
PS4, Line 362: where l_orderkey < (select min(l_partkey) from
Was removing abs intentional?



-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 4
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zihao Ye <ey...@163.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 11:46:53 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/20891 )

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................


Patch Set 2: Code-Review+1

(2 comments)

http://gerrit.cloudera.org:8080/#/c/20891/2/tests/query_test/test_join_queries.py
File tests/query_test/test_join_queries.py:

http://gerrit.cloudera.org:8080/#/c/20891/2/tests/query_test/test_join_queries.py@116
PS2, Line 116:   def test_nested_loop_joins_with_min_max_runtime_filter(self, vector, unique_database):
Checked the original patch that added the bug and it did add a test with nested loop join + min max filter:
https://gerrit.cloudera.org/#/c/17706/34/testdata/workloads/functional-query/queries/QueryTest/overlap_min_max_filters_on_sorted_columns.test

Do you know the difference, so why does this query trigger the issue, but not the old query?


http://gerrit.cloudera.org:8080/#/c/20891/2/tests/query_test/test_join_queries.py@120
PS2, Line 120:         "CREATE TABLE {0} (id int) PARTITIONED BY(dt string) "
optional: I think that this would be more readable if it was moved to an existing .test file or a now one.=



-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 2
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 15 Jan 2024 12:58:01 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Zihao Ye (Code Review)" <ge...@cloudera.org>.
Hello Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/20891

to look at the new patch set (#3).

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................

IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join

This patch fixes an issue reported by IMPALA-12582, where enabling
MIN_MAX RuntimeFilter for a specific query would cause the executor to
crash. The direct cause of the crash was an out-of-bounds access to
input_vals in the ScalarFnCall::InterpretEval() function, but the root
cause was actually due to the related ScalarExprEvaluator not invoking
the Open() function.

Testing:
 - Added new E2E test case about this issue.
 - Added new test dimension of DISABLE_CODEGEN to improve code coverage.

Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
---
M be/src/exec/nested-loop-join-builder.cc
M testdata/workloads/functional-query/queries/QueryTest/overlap_min_max_filters_on_partition_columns.test
M testdata/workloads/functional-query/queries/QueryTest/overlap_min_max_filters_on_sorted_columns.test
M tests/query_test/test_runtime_filters.py
4 files changed, 23 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/20891/3
-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 3
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>

[Impala-ASF-CR] IMPALA-12582: Fix crash when enabling MIN MAX RuntimeFilter in Nested Loop Join

Posted by "Zihao Ye (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/20891

to look at the new patch set (#2).

Change subject: IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join
......................................................................

IMPALA-12582: Fix crash when enabling MIN_MAX RuntimeFilter in Nested Loop Join

IMPALA-12582 reported an issue where enabling MIN_MAX RuntimeFilter for
a specific query would cause the executor to crash. The direct cause of
the crash was an out-of-bounds access to input_vals in the
ScalarFnCall::InterpretEval() function, but the root cause was actually
due to the related ScalarExprEvaluator not invoking the Open() function.
due to the related ScalarExprEvaluator not invoking Open(). This patch
fix this issue and adds corresponding test cases in
test_join_queries.py.

Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
---
M be/src/exec/nested-loop-join-builder.cc
M tests/query_test/test_join_queries.py
2 files changed, 21 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/20891/2
-- 
To view, visit http://gerrit.cloudera.org:8080/20891
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iba951796d52f109c419587c444840adbb2d44f5d
Gerrit-Change-Number: 20891
Gerrit-PatchSet: 2
Gerrit-Owner: Zihao Ye <ey...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>