[jira] [Assigned] (HUDI-9612) Improve schema resolver performance by skipping delete-only files

["Timothy Brown (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Timothy Brown reassigned HUDI-9612:
-----------------------------------

    Assignee: Timothy Brown

> Improve schema resolver performance by skipping delete-only files
> -----------------------------------------------------------------
>
>                 Key: HUDI-9612
>                 URL: https://issues.apache.org/jira/browse/HUDI-9612
>             Project: Apache Hudi
>          Issue Type: Improvement
>            Reporter: Timothy Brown
>            Assignee: Timothy Brown
>            Priority: Major
>
> The TableSchemaResolver can have varied performance depending on the last commit to the table. The code will currently try to inspect log files with only deletes for example which will not yield a schema and then we must inspect another file. In the worst case, you can iterate through all the files and still not find a valid schema.
> This can be updated to first ensure the commit has a non-zero number of inserts or updates, then inspect files that have non-zero inserts or updates only.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)