[jira] [Created] (HUDI-3594) Support standard Spark functions in Filter Exprs in Data Skipping

["Alexey Kudinkin (Jira)" <ji...@apache.org>]

Alexey Kudinkin created HUDI-3594:
-------------------------------------

             Summary: Support standard Spark functions in Filter Exprs in Data Skipping
                 Key: HUDI-3594
                 URL: https://issues.apache.org/jira/browse/HUDI-3594
             Project: Apache Hudi
          Issue Type: Bug
            Reporter: Alexey Kudinkin
            Assignee: Alexey Kudinkin


As part of this effort we're planning to (at the very least) support a suite of standard Spark functions when evaluating Data Filtering expressions w/in Data Skipping flow, for ex: when user is issuing a following query 

 
{code:java}
SELECT ... WHERE date_format(ts, 'dd-mm-yyyy') > '01-01-2022'
{code}
We're able to relate such query to our Column Stats Index appropriately, therefore being able to do Data Skipping not only on the "raw" columns, but also upon simple derivative expressions on top of them (like standard function calls){*}{*}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)