[GitHub] [incubator-iceberg] rdblue commented on issue #412: Use null counts in metrics evaluators

[GitBox <gi...@apache.org>]

rdblue commented on issue #412: Use null counts in metrics evaluators
URL: https://github.com/apache/incubator-iceberg/pull/412#issuecomment-524914156
 
 
   > Is the current implementation of eq in StrictMetricsEvaluator safe? What if we have truncated min/max values?
   
   I think the bounds check is safe with truncated bounds. The check is that the literal value is equal to lower and upper bounds, so we would know that there is only one value that is equal to the filter's literal. That can only happen when the bounds have not been truncated because the truncated upper bound is never equal to the truncated lower bound unless the value was already smaller than the truncation length.
   
   Here's an example: Say the real min and max are ["aa", "aaaa"] and we are truncating to 2 codepoints. Then the bounds will be ["aa", "ab"]. Because there is no value equal to both bounds, `eq` will never return `ROWS_MUST_MATCH`. Alternatively, if there is just one value, "aa", then both min/max and truncated bounds will be "aa". In that case, `eq` will detect that "aa" is equal to both bounds and return that the predicate is guaranteed to be true by the bounds.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org