[jira] [Commented] (HIVE-21599) Parquet predicate pushdown on partition columns may cause wrong result if files contain partition columns

["Soumyakanti Das (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-21599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17631897#comment-17631897 ] 

Soumyakanti Das commented on HIVE-21599:
----------------------------------------

The new patch works by removing the partition columns from the Parquet schema. When Partition columns are not present in the schema, filter predicates with partition columns are not pushed down.

> Parquet predicate pushdown on partition columns may cause wrong result if files contain partition columns
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-21599
>                 URL: https://issues.apache.org/jira/browse/HIVE-21599
>             Project: Hive
>          Issue Type: Improvement
>          Components: Query Planning
>            Reporter: Vineet Garg
>            Assignee: Soumyakanti Das
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: HIVE-21599.1.patch
>
>          Time Spent: 3h
>  Remaining Estimate: 0h
>
> Filter predicates are pushed to Table Scan (to be pushed to and used by storage handler/input format). Such predicates could consist of partition columns which are of no use to storage handler  or input formats. Therefore it should be removed from TS filter expression.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)