You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Davis Zhang (Jira)" <ji...@apache.org> on 2024/11/08 01:47:00 UTC
[jira] [Assigned] (HUDI-8491) Fix concurrent schema evolution conflict bug
[ https://issues.apache.org/jira/browse/HUDI-8491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Davis Zhang reassigned HUDI-8491:
---------------------------------
Assignee: Davis Zhang
> Fix concurrent schema evolution conflict bug
> --------------------------------------------
>
> Key: HUDI-8491
> URL: https://issues.apache.org/jira/browse/HUDI-8491
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Davis Zhang
> Assignee: Davis Zhang
> Priority: Major
>
> For schema conflict resolution, previously we assume all instances with type COMMIT or DELTA_COMMIT comes with a valid schema. This is not true as there can be cases where hoodie streamer makes empty insert commits just for purpose of saving some commit metadata like the checkpoint offset while no data is written to the hudi table (maybe all messages fetched from a data source like kafka turns out to be not useful in that batch). In such a case, the commit metadata does not necessarily comes with a valid table schema ("schema" field is ""null"")
> The change accommodates for this corner case.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)