You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@parquet.apache.org by "Gabor Szadovszky (Jira)" <ji...@apache.org> on 2023/02/15 11:00:00 UTC

[jira] [Resolved] (PARQUET-2244) Dictionary filter may skip row-groups incorrectly when evaluating notIn

     [ https://issues.apache.org/jira/browse/PARQUET-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gabor Szadovszky resolved PARQUET-2244.
---------------------------------------
    Resolution: Fixed

> Dictionary filter may skip row-groups incorrectly when evaluating notIn
> -----------------------------------------------------------------------
>
>                 Key: PARQUET-2244
>                 URL: https://issues.apache.org/jira/browse/PARQUET-2244
>             Project: Parquet
>          Issue Type: Bug
>          Components: parquet-mr
>    Affects Versions: 1.12.2
>            Reporter: Yujiang Zhong
>            Assignee: Yujiang Zhong
>            Priority: Major
>
> Dictionary filter may skip row-groups incorrectly when evaluating `notIn` on optional columns with null values. Here is an example:
> Say there is a optional column `c1` with all pages dict encoded, `c1` has and only has two distinct values: ['foo', null],  and the predicate is  `c1 not in ('foo', 'bar')`. 
> Now dictionary filter may skip this row-group that is actually should not be skipped, because there are nulls in the column.
>  
> This is a bug similar to #1510.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)