You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@iceberg.apache.org by Anton Okolnychyi <ao...@apple.com.INVALID> on 2019/08/12 20:29:09 UTC

InclusiveMetricsEvaluator null counts

Hey folks,

A quick question: InclusiveMetricsEvaluator doesn't check null counts in qt/gtEq/lt/ltEq predicates as opposed to ParquetMetricsRowGroupFilter, meaning files containing only null values will always match. Am I correct that it's on purpose and we expect query engines to always infer isNotNull whenever we have qt/gtEq/lt/ltEq?

Thanks,
Anton

Re: InclusiveMetricsEvaluator null counts

Posted by Ryan Blue <rb...@netflix.com.INVALID>.

I think that's an oversight. Files that contain all null values should not
match inequalities.

On Mon, Aug 12, 2019 at 1:29 PM Anton Okolnychyi
<ao...@apple.com.invalid> wrote:

> Hey folks,
>
> A quick question: InclusiveMetricsEvaluator doesn't check null counts in
> qt/gtEq/lt/ltEq predicates as opposed to ParquetMetricsRowGroupFilter,
> meaning files containing only null values will always match. Am I correct
> that it's on purpose and we expect query engines to always infer isNotNull
> whenever we have qt/gtEq/lt/ltEq?
>
> Thanks,
> Anton



-- 
Ryan Blue
Software Engineer
Netflix