[jira] [Assigned] (HUDI-4734) Add table config change validation in deltastreamer

["Raymond Xu (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-4734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raymond Xu reassigned HUDI-4734:
--------------------------------

    Assignee: Vamshi Gudavarthi  (was: sivabalan narayanan)

> Add table config change validation in deltastreamer
> ---------------------------------------------------
>
>                 Key: HUDI-4734
>                 URL: https://issues.apache.org/jira/browse/HUDI-4734
>             Project: Apache Hudi
>          Issue Type: Improvement
>          Components: deltastreamer
>            Reporter: sivabalan narayanan
>            Assignee: Vamshi Gudavarthi
>            Priority: Major
>             Fix For: 0.12.1
>
>
> looks like we are missing proper table config validation in deltastreamer. 
> {code:java}
>    if (fs.exists(new Path(cfg.targetBasePath))) {
>         HoodieTableMetaClient meta =
>             HoodieTableMetaClient.builder().setConf(new Configuration(fs.getConf())).setBasePath(cfg.targetBasePath).setLoadActiveTimelineOnLoad(false).build();
>         tableType = meta.getTableType();
>         // This will guarantee there is no surprise with table type
>         ValidationUtils.checkArgument(tableType.equals(HoodieTableType.valueOf(cfg.tableType)),
>             "Hoodie table is of type " + tableType + " but passed in CLI argument is " + cfg.tableType);
>         // Load base file format
>         // This will guarantee there is no surprise with base file type
>         String baseFileFormat = meta.getTableConfig().getBaseFileFormat().toString();
>         ValidationUtils.checkArgument(baseFileFormat.equals(cfg.baseFileFormat) || cfg.baseFileFormat == null,
>             "Hoodie table's base file format is of type " + baseFileFormat + " but passed in CLI argument is "
>                 + cfg.baseFileFormat);
>         cfg.baseFileFormat = baseFileFormat;
>         this.cfg.baseFileFormat = baseFileFormat;
>       } else {
>         tableType = HoodieTableType.valueOf(cfg.tableType);
>         if (cfg.baseFileFormat == null) {
>           cfg.baseFileFormat = "PARQUET"; // default for backward compatibility
>         }
>       } {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)