You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "lvyanquan (Jira)" <ji...@apache.org> on 2023/04/01 10:22:00 UTC
[jira] [Updated] (HUDI-6012) delete base path when failed to run bootstrap procedure
[ https://issues.apache.org/jira/browse/HUDI-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
lvyanquan updated HUDI-6012:
----------------------------
Description:
[run_bootstrap](https://hudi.apache.org/docs/next/procedures#run_bootstrap) procedure is called like this
{code:java}
call run_bootstrap(table => 'test_hudi_table', table_type => 'COPY_ON_WRITE', bootstrap_path => 'hdfs://ns1/hive/warehouse/hudi.db/test_hudi_table', base_path => 'hdfs://ns1//tmp/hoodie/test_hudi_table', rowKey_field => 'id', partition_path_field => 'dt'); {code}
some exceptional cases this procedure will fail, for example, bootstrap_path is not existed or empty. The `base_path` in HDFS was still remained with `.hoodie` directory.
Though we can still rerun bootstrap procedure and pass `bootstrap_overwrite` parameter, it's better to clean this path that we created after failure.
was:
when we failed to `run_bootstrap` procedure, the `base_path` was remained with `.hoodie` directory.
Though we can still rerun bootstrap procedure and pass `bootstrap_overwrite` parameter, it's better to clean this path that we created after failure.
> delete base path when failed to run bootstrap procedure
> -------------------------------------------------------
>
> Key: HUDI-6012
> URL: https://issues.apache.org/jira/browse/HUDI-6012
> Project: Apache Hudi
> Issue Type: Improvement
> Components: bootstrap
> Reporter: lvyanquan
> Priority: Major
>
> [run_bootstrap](https://hudi.apache.org/docs/next/procedures#run_bootstrap) procedure is called like this
> {code:java}
> call run_bootstrap(table => 'test_hudi_table', table_type => 'COPY_ON_WRITE', bootstrap_path => 'hdfs://ns1/hive/warehouse/hudi.db/test_hudi_table', base_path => 'hdfs://ns1//tmp/hoodie/test_hudi_table', rowKey_field => 'id', partition_path_field => 'dt'); {code}
> some exceptional cases this procedure will fail, for example, bootstrap_path is not existed or empty. The `base_path` in HDFS was still remained with `.hoodie` directory.
> Though we can still rerun bootstrap procedure and pass `bootstrap_overwrite` parameter, it's better to clean this path that we created after failure.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)