You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@impala.apache.org by "Steve Carlin (Code Review)" <ge...@cloudera.org> on 2021/03/18 14:22:49 UTC

[Impala-ASF-CR] IMPALA-10593: Skip runtime filter for outer joins...

Steve Carlin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17200


Change subject: IMPALA-10593: Skip runtime filter for outer joins...
......................................................................

IMPALA-10593: Skip runtime filter for outer joins...

...when Expr not constant after null substitution.

Currently there is code that asserts that an Expr is not constant after
substituting SlotRefs with constant nulls.

A third party tool needs this restriction to be weakened.  In a case where
an Expr is checked and the Expr is not constant even after substituting
nulls, the result will be to not generate a runtime filter for that Expr.

Change-Id: I46462e2030731d97c4c88e364148c0093c025ab3
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java
2 files changed, 8 insertions(+), 1 deletion(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/00/17200/1
-- 
To view, visit http://gerrit.cloudera.org:8080/17200
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I46462e2030731d97c4c88e364148c0093c025ab3
Gerrit-Change-Number: 17200
Gerrit-PatchSet: 1
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>

[Impala-ASF-CR] IMPALA-10593: Conditionally skip runtime filter for outer joins

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.

Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17200 )

Change subject: IMPALA-10593: Conditionally skip runtime filter for outer joins
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/8454/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/17200
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I46462e2030731d97c4c88e364148c0093c025ab3
Gerrit-Change-Number: 17200
Gerrit-PatchSet: 4
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Sun, 28 Mar 2021 00:04:14 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10593: Conditionally skip runtime filter for outer joins

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.

Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17200 )

Change subject: IMPALA-10593: Conditionally skip runtime filter for outer joins
......................................................................


Patch Set 5: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/17200
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I46462e2030731d97c4c88e364148c0093c025ab3
Gerrit-Change-Number: 17200
Gerrit-PatchSet: 5
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Sat, 27 Mar 2021 23:59:10 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10593: Conditionally skip runtime filter for outer joins

Posted by "Steve Carlin (Code Review)" <ge...@cloudera.org>.

Steve Carlin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17200 )

Change subject: IMPALA-10593: Conditionally skip runtime filter for outer joins
......................................................................


Patch Set 1:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/17200/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/17200/1//COMMIT_MSG@7
PS1, Line 7: ...
> nit: instead of ... maybe reword it as 'Conditionally skip runtime filter f
Done


http://gerrit.cloudera.org:8080/#/c/17200/1//COMMIT_MSG@9
PS1, Line 9: ...when Expr not constant after null substitution.
> then this can be removed.
Done


http://gerrit.cloudera.org:8080/#/c/17200/1//COMMIT_MSG@14
PS1, Line 14: A third party tool needs
> nit: For external FE (to be consistent with other patches with similar exte
Done


http://gerrit.cloudera.org:8080/#/c/17200/1//COMMIT_MSG@16
PS1, Line 16: nulls, the result will be to not generate a runtime filter for that Expr.
> Add a 'Testing:' section here and provide the test query and indicate that 
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/17200
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I46462e2030731d97c4c88e364148c0093c025ab3
Gerrit-Change-Number: 17200
Gerrit-PatchSet: 1
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Wed, 24 Mar 2021 00:04:29 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-10593: Conditionally skip runtime filter for outer joins

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.

Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17200 )

Change subject: IMPALA-10593: Conditionally skip runtime filter for outer joins
......................................................................

IMPALA-10593: Conditionally skip runtime filter for outer joins

Currently there is code that asserts that an Expr is not constant after
substituting SlotRefs with constant nulls.

For External FE, this restriction to be weakened.  In a case where
an Expr is checked and the Expr is not constant even after substituting
nulls, the result will be to not generate a runtime filter for that Expr.

Testing:

Manually tested with this query in the External FE:

select id, int_col, year, month from alltypessmall s
where s.int_col = (select count(*) from alltypestiny t where s.id = t.id)
order by id

Change-Id: I46462e2030731d97c4c88e364148c0093c025ab3
Reviewed-on: http://gerrit.cloudera.org:8080/17200
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java
2 files changed, 8 insertions(+), 1 deletion(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/17200
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I46462e2030731d97c4c88e364148c0093c025ab3
Gerrit-Change-Number: 17200
Gerrit-PatchSet: 6
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>

[Impala-ASF-CR] IMPALA-10593: Conditionally skip runtime filter for outer joins

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.

Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17200 )

Change subject: IMPALA-10593: Conditionally skip runtime filter for outer joins
......................................................................


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7015/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/17200
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I46462e2030731d97c4c88e364148c0093c025ab3
Gerrit-Change-Number: 17200
Gerrit-PatchSet: 5
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Sat, 27 Mar 2021 23:59:11 +0000
Gerrit-HasComments: No