You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Vinoth Chandar (Jira)" <ji...@apache.org> on 2024/05/02 01:15:00 UTC
[jira] [Updated] (HUDI-7538) Consolidate the CDC Formats (changelog format, RFC-51)
[ https://issues.apache.org/jira/browse/HUDI-7538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinoth Chandar updated HUDI-7538:
---------------------------------
Description:
For sake of more consistency, we need to consolidate the the changelog mode (currently supported for Flink MoR) and RFC-51 based CDC feature which is a debezium style change log (currently supported for CoW for Spark/Flink)
|Format Name|CDC Source Required|Resource Cost(writer)|Resource Cost(reader)|Friendly to Streaming|
|CDC|*No*|low/high|low/high (based on logging modes we choose)|No (the debezium style output is not what Flink needs for e.g)|
|Changelog|Yes|low|low|Yes|
This proposal is to converge onto "CDC" as the path going forward, with the following changes to incorporate.
> Consolidate the CDC Formats (changelog format, RFC-51)
> ------------------------------------------------------
>
> Key: HUDI-7538
> URL: https://issues.apache.org/jira/browse/HUDI-7538
> Project: Apache Hudi
> Issue Type: Improvement
> Components: storage-management
> Reporter: Vinoth Chandar
> Assignee: Vinoth Chandar
> Priority: Major
> Labels: hudi-1.0.0-beta2
> Fix For: 1.0.0
>
>
> For sake of more consistency, we need to consolidate the the changelog mode (currently supported for Flink MoR) and RFC-51 based CDC feature which is a debezium style change log (currently supported for CoW for Spark/Flink)
>
> |Format Name|CDC Source Required|Resource Cost(writer)|Resource Cost(reader)|Friendly to Streaming|
> |CDC|*No*|low/high|low/high (based on logging modes we choose)|No (the debezium style output is not what Flink needs for e.g)|
> |Changelog|Yes|low|low|Yes|
> This proposal is to converge onto "CDC" as the path going forward, with the following changes to incorporate.
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)