[jira] [Closed] (HUDI-3739) Fix translation of isNotNull predicates in Data Skipping

["Alexey Kudinkin (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-3739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alexey Kudinkin closed HUDI-3739.
---------------------------------
    Resolution: Fixed

> Fix translation of isNotNull predicates in Data Skipping
> --------------------------------------------------------
>
>                 Key: HUDI-3739
>                 URL: https://issues.apache.org/jira/browse/HUDI-3739
>             Project: Apache Hudi
>          Issue Type: Bug
>            Reporter: Alexey Kudinkin
>            Assignee: Alexey Kudinkin
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 0.11.0
>
>
> Right now isNotNull predicates are translated incorrectly 
> {code:java}
> // Filter "colA is not null"
> // Translates to "colA_num_nulls = 0" for index lookup
> case IsNotNull(attribute: AttributeReference) =>
>   getTargetIndexedColumnName(attribute, indexSchema)
>     .map(colName => EqualTo(genColNumNullsExpr(colName), Literal(0))) {code}
> Instead of "colA_num_nulls = 0" the condition should be "colA_num_nulls != colA_numRecords"



--
This message was sent by Atlassian Jira
(v8.20.1#820001)