[I] Expression index is used for filters on a different function of the same column, returning wrong results [hudi]

["yihua (via GitHub)" <gi...@apache.org>]

yihua opened a new issue, #20255:
URL: https://github.com/apache/hudi/issues/20255

   With an expression index on a function of a column, data skipping also uses that index for filters on a different function of the same column. Example: with `create index idx on t using column_stats(city) options(expr='lower')`, the query `select * from t where upper(city) = 'CITY_4242'` returns no rows (1,224 with `hoodie.enable.data.skipping=false`), and `where length(city) = 9` fails with `ClassCastException: String cannot be cast to Integer`.
   
   Cause: `ExpressionIndexSupport.extractQueryAndLiterals` unwraps any `UnaryExpression` over the index's source column without checking that it is the index's function. Proposal: match the filter's function against the index definition before using the index, and fall back to column stats otherwise.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Expression index is used for filters on a different function of the same column, returning wrong results [hudi]

["yihua (via GitHub)" <gi...@apache.org>]

yihua commented on issue #20255:
URL: https://github.com/apache/hudi/issues/20255#issuecomment-6076515554

   Fixed by #20256.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org