You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Sagar Sumit (Jira)" <ji...@apache.org> on 2022/04/05 03:11:00 UTC
[jira] [Updated] (HUDI-3739) Fix translation of isNotNull predicates in Data Skipping
[ https://issues.apache.org/jira/browse/HUDI-3739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sagar Sumit updated HUDI-3739:
------------------------------
Reviewers: Sagar Sumit
> Fix translation of isNotNull predicates in Data Skipping
> --------------------------------------------------------
>
> Key: HUDI-3739
> URL: https://issues.apache.org/jira/browse/HUDI-3739
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Alexey Kudinkin
> Assignee: Alexey Kudinkin
> Priority: Blocker
> Fix For: 0.11.0
>
>
> Right now isNotNull predicates are translated incorrectly
> {code:java}
> // Filter "colA is not null"
> // Translates to "colA_num_nulls = 0" for index lookup
> case IsNotNull(attribute: AttributeReference) =>
> getTargetIndexedColumnName(attribute, indexSchema)
> .map(colName => EqualTo(genColNumNullsExpr(colName), Literal(0))) {code}
> Instead of "colA_num_nulls = 0" the condition should be "colA_num_nulls != colA_numRecords"
--
This message was sent by Atlassian Jira
(v8.20.1#820001)