[jira] [Updated] (HUDI-8001) Insert overwrite failed due to missing 'path' property when using Spark 3.5.1 and Hudi 1.0.0

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-8001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated HUDI-8001:
---------------------------------
    Labels: pull-request-available  (was: )

> Insert overwrite failed due to missing 'path' property when using Spark 3.5.1 and Hudi 1.0.0
> --------------------------------------------------------------------------------------------
>
>                 Key: HUDI-8001
>                 URL: https://issues.apache.org/jira/browse/HUDI-8001
>             Project: Apache Hudi
>          Issue Type: Bug
>            Reporter: Ma Jian
>            Priority: Major
>              Labels: pull-request-available
>
> The issue with Spark 3.5.1 arises because the {{InsertIntoHoodieTableCommand}} chain calls the initialization of the {{HoodieFileIndex}} class. For v1 tables, the path is stored in {{{}CatalogTable#CatalogStorageFormat#storageProperties{}}}, but not in {{{}CatalogTable#properties{}}}. When Spark reloads the table, it removes the path key from {{{}CatalogTable#CatalogStorageFormat#storageProperties{}}}. Consequently, {{InsertIntoHoodieTableCommand}} in Hudi cannot retrieve the path from either {{CatalogTable#CatalogStorageFormat#storageProperties}} or {{CatalogTable#properties}} during {{{}deduceOverwriteConfig{}}}. This absence of the path key in {{combinedOpts}} leads to an error when initializing {{{}HoodieFileIndex{}}}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)