You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Ethan Guo (Jira)" <ji...@apache.org> on 2024/06/07 01:12:00 UTC

[jira] [Closed] (HUDI-7237) Minor Improvements to Schema Handling in Delta Sync

     [ https://issues.apache.org/jira/browse/HUDI-7237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ethan Guo closed HUDI-7237.
---------------------------
    Resolution: Fixed

> Minor Improvements to Schema Handling in Delta Sync
> ---------------------------------------------------
>
>                 Key: HUDI-7237
>                 URL: https://issues.apache.org/jira/browse/HUDI-7237
>             Project: Apache Hudi
>          Issue Type: Improvement
>            Reporter: Timothy Brown
>            Priority: Minor
>              Labels: pull-request-available
>             Fix For: 0.15.0, 1.0.0
>
>
> There are a two minor items that we have run into running DeltaStreamer in production.
> 1. The number of times the schema is fetched is more than it needs to be and can put unnecessary load on schema providers or increase file system reads
> 2. SchemaProviders that return null target schemas on empty batches cause null schema values in commits leading to unexpected issues later
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)