[jira] [Updated] (HUDI-3594) Support standard Spark functions in Filter Exprs in Data Skipping

["Alexey Kudinkin (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alexey Kudinkin updated HUDI-3594:
----------------------------------
    Epic Link: HUDI-1822

> Support standard Spark functions in Filter Exprs in Data Skipping
> -----------------------------------------------------------------
>
>                 Key: HUDI-3594
>                 URL: https://issues.apache.org/jira/browse/HUDI-3594
>             Project: Apache Hudi
>          Issue Type: Bug
>            Reporter: Alexey Kudinkin
>            Assignee: Alexey Kudinkin
>            Priority: Blocker
>
> As part of this effort we're planning to (at the very least) support a suite of standard Spark functions when evaluating Data Filtering expressions w/in Data Skipping flow, for ex: when user is issuing a following query 
>  
> {code:java}
> SELECT ... WHERE date_format(ts, 'dd-mm-yyyy') > '01-01-2022'
> {code}
> We're able to relate such query to our Column Stats Index appropriately, therefore being able to do Data Skipping not only on the "raw" columns, but also upon simple derivative expressions on top of them (like standard function calls){*}{*}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)