You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Ethan Guo (Jira)" <ji...@apache.org> on 2024/06/07 01:12:00 UTC
[jira] [Closed] (HUDI-7237) Minor Improvements to Schema Handling in Delta Sync
[ https://issues.apache.org/jira/browse/HUDI-7237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ethan Guo closed HUDI-7237.
---------------------------
Resolution: Fixed
> Minor Improvements to Schema Handling in Delta Sync
> ---------------------------------------------------
>
> Key: HUDI-7237
> URL: https://issues.apache.org/jira/browse/HUDI-7237
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: Timothy Brown
> Priority: Minor
> Labels: pull-request-available
> Fix For: 0.15.0, 1.0.0
>
>
> There are a two minor items that we have run into running DeltaStreamer in production.
> 1. The number of times the schema is fetched is more than it needs to be and can put unnecessary load on schema providers or increase file system reads
> 2. SchemaProviders that return null target schemas on empty batches cause null schema values in commits leading to unexpected issues later
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)