You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Aman Sinha (Jira)" <ji...@apache.org> on 2020/12/28 00:14:00 UTC
[jira] [Resolved] (IMPALA-936) Generalize assignment of predicates
inside outer-joined inline views.
[ https://issues.apache.org/jira/browse/IMPALA-936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aman Sinha resolved IMPALA-936.
-------------------------------
Resolution: Cannot Reproduce
This was likely fixed in an earlier release. I get the following plan on 4.0-SNAPSHOT which shows that the round(int_col) predicate was pushed to the scan below the outer join:
{noformat}
Query: explain select b.x from functional.alltypes a
left outer join (select id, round(int_col) x from functional.alltypessmall) b
on (a.id = b.id)
where b.x > 10
+------------------------------------------------------------+
| Explain String |
+------------------------------------------------------------+
| Max Per-Host Resource Reservation: Memory=1.98MB Threads=5 |
| Per-Host Resource Estimates: Memory=162MB |
| Codegen disabled by planner |
| |
| PLAN-ROOT SINK |
| | |
| 04:EXCHANGE [UNPARTITIONED] |
| | |
| 02:HASH JOIN [LEFT OUTER JOIN, BROADCAST] |
| | hash predicates: a.id = id |
| | other predicates: round(int_col) > 10 |
| | row-size=12B cardinality=7.30K |
| | |
| |--03:EXCHANGE [BROADCAST] |
| | | |
| | 01:SCAN HDFS [functional.alltypessmall] |
| | HDFS partitions=4/4 files=4 size=6.32KB |
| | predicates: round(int_col) > 10 |
| | row-size=8B cardinality=10 |
| | |
| 00:SCAN HDFS [functional.alltypes a] |
| HDFS partitions=24/24 files=24 size=478.45KB |
| row-size=4B cardinality=7.30K |
+------------------------------------------------------------+
{noformat}
> Generalize assignment of predicates inside outer-joined inline views.
> ---------------------------------------------------------------------
>
> Key: IMPALA-936
> URL: https://issues.apache.org/jira/browse/IMPALA-936
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 1.3
> Reporter: Alexander Behm
> Priority: Minor
>
> In some cases we fail to assign predicates inside outer-joined inline views, although it would be safe to do so.
> Consider these examples and their plans:
> {code}
> select b.x from functional.alltypes a
> left outer join (select id, int_col x from functional.alltypessmall) b
> on (a.id = b.id)
> where b.x > 10
> +-----------------------------------------------------------+
> | Explain String |
> +-----------------------------------------------------------+
> | Estimated Per-Host Requirements: Memory=160.00MB VCores=2 |
> | |
> | 04:EXCHANGE [PARTITION=UNPARTITIONED] |
> | | |
> | 02:HASH JOIN [LEFT OUTER JOIN, BROADCAST] |
> | | hash predicates: a.id = id |
> | | other predicates: int_col > 10 |
> | | |
> | |--03:EXCHANGE [BROADCAST] |
> | | | |
> | | 01:SCAN HDFS [functional.alltypessmall] |
> | | partitions=4/4 size=6.32KB |
> | | predicates: functional.alltypessmall.int_col > 10 |
> | | |
> | 00:SCAN HDFS [functional.alltypes a] |
> | partitions=24/24 size=478.45KB |
> +-----------------------------------------------------------+
> {code}
> Here the predicate is picked up by getBoundPredicates() inside the scan of alltypessmall because there is a value transfer between slots.
> Notice that for correctness the predicate is also evaluated at the join node.
> In the following query, the predicate is not assigned in the scan of alltypessmall.
> The reason is that there is no direct value transfer between slots, so getBoundPredicates() doesn't pick up the predicate.
> {code}
> select b.x from functional.alltypes a
> left outer join (select id, round(int_col) x from functional.alltypessmall) b
> on (a.id = b.id)
> where b.x > 10
> +-----------------------------------------------------------+
> | Explain String |
> +-----------------------------------------------------------+
> | Estimated Per-Host Requirements: Memory=160.00MB VCores=2 |
> | |
> | 04:EXCHANGE [PARTITION=UNPARTITIONED] |
> | | |
> | 02:HASH JOIN [LEFT OUTER JOIN, BROADCAST] |
> | | hash predicates: a.id = id |
> | | other predicates: round(int_col) > 10 |
> | | |
> | |--03:EXCHANGE [BROADCAST] |
> | | | |
> | | 01:SCAN HDFS [functional.alltypessmall] |
> | | partitions=4/4 size=6.32KB |
> | | |
> | 00:SCAN HDFS [functional.alltypes a] |
> | partitions=24/24 size=478.45KB |
> +-----------------------------------------------------------+
> {code}
> One possible way to solve this issue could be to relax Analyzer.canEvalPredicate(), s.t. the predicate is 'pushed' inside the inline view while not marking it as assigned in Planner.createInlineViewPlan().
--
This message was sent by Atlassian Jira
(v8.3.4#803005)