You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@impala.apache.org by "Tim Armstrong (Jira)" <ji...@apache.org> on 2020/05/21 03:31:00 UTC

[jira] [Resolved] (IMPALA-8834) Investigate enabling safe version of OPTIMIZE_PARTITION_KEY_SCANS by default

     [ https://issues.apache.org/jira/browse/IMPALA-8834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tim Armstrong resolved IMPALA-8834.
-----------------------------------
    Fix Version/s: Impala 4.0
       Resolution: Fixed

> Investigate enabling safe version of OPTIMIZE_PARTITION_KEY_SCANS by default
> ----------------------------------------------------------------------------
>
>                 Key: IMPALA-8834
>                 URL: https://issues.apache.org/jira/browse/IMPALA-8834
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Backend
>            Reporter: Tim Armstrong
>            Assignee: Tim Armstrong
>            Priority: Major
>              Labels: perf
>             Fix For: Impala 4.0
>
>
> Just an idea I had while updating the docs. We already have the logic in the planner to determine when a partition key scan has "distinct" semantics - i.e. you obtain the correct results so long as the scan returns 0 rows for a partition when there are no rows present, and at least one row when there is a row present (but the exact number doesn't affect correctness).
> We could push this knowledge down into the scan nodes and have them terminate early. This would be quite efficient if file handles and footers were already cached and greatly reduce the number of rows flowing through the rest of the plan.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)