You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Lokesh Jain (Jira)" <ji...@apache.org> on 2023/04/03 12:40:00 UTC
[jira] [Resolved] (HUDI-5740) Refactor Deltastreamer and schema providers to use HoodieConfig/ConfigProperty
[ https://issues.apache.org/jira/browse/HUDI-5740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lokesh Jain resolved HUDI-5740.
-------------------------------
> Refactor Deltastreamer and schema providers to use HoodieConfig/ConfigProperty
> ------------------------------------------------------------------------------
>
> Key: HUDI-5740
> URL: https://issues.apache.org/jira/browse/HUDI-5740
> Project: Apache Hudi
> Issue Type: Improvement
> Components: configs, deltastreamer
> Reporter: Jonathan Vexler
> Assignee: Lokesh Jain
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.14.0
>
>
> The configs in the following classes are implemented not using HoodieConfig, making it impossible to be surfaced on the Configurations page. We need to refactor the code so that each config property is implemented using ConfigProperty in a corresponding new HoodieConfig class. Refer to HoodieArchivalConfig for existing implementation of configs.
>
> InitialCheckPointProvider
> HoodieDeltaStreamer
> HoodieMultiTableDeltaStreamer
> FilebasedSchemaProvider
> HiveSchemaProvider
> JdbcbasedSchemaProvider
> ProtoClassBasedSchemaProvider
> SchemaPostProcessor
> SchemaRegistryProvider
> SparkAvroPostProcessor
> DropColumnSchemaPostProcessor
> BaseSchemaPostProcessorConfig
> KafkaOffsetPostProcessor
> SanitizationUtils
> Also 'hoodie.deltastreamer.multiwriter.source.checkpoint.id' in HoodieWriteConfig
--
This message was sent by Atlassian Jira
(v8.20.10#820010)