[jira] [Created] (HUDI-4447) Hive Sync fails fails when performing delete table data operation

["rex xiong (Jira)" <ji...@apache.org>]

rex xiong created HUDI-4447:
-------------------------------

             Summary: Hive Sync fails fails when performing delete table data operation
                 Key: HUDI-4447
                 URL: https://issues.apache.org/jira/browse/HUDI-4447
             Project: Apache Hudi
          Issue Type: Bug
          Components: meta-sync
         Environment: Spark3.2.1 & Hudi 0.11.1
            Reporter: rex xiong
            Assignee: rex xiong
             Fix For: 0.12.0


 currently,  sync meta would fail if performing delete table data operation, because sync without setting database name/fields/extractor class, etc.This causes two problems：

1）sync schema will recreate table in default database

2）sync will use wrong extractor class(HiveStylePartitionValueExtractor) when syncing non partitioned table with metadata table enabled

 
{code:java}
spark-sql> delete from test_db.hudi_mor_none_part_table_321_0111 where id=1;
org.apache.hudi.exception.HoodieException: Could not sync using the meta sync class org.apache.hudi.hive.HiveSyncTool
    at org.apache.hudi.sync.common.util.SyncUtilHelpers.runHoodieMetaSync(SyncUtilHelpers.java:61)
    at org.apache.hudi.HoodieSparkSqlWriter$.$anonfun$metaSync$2(HoodieSparkSqlWriter.scala:626)
    at org.apache.hudi.HoodieSparkSqlWriter$.$anonfun$metaSync$2$adapted(HoodieSparkSqlWriter.scala:625)
    at scala.collection.mutable.HashSet.foreach(HashSet.scala:79)
....
Caused by: org.apache.hudi.exception.HoodieException: Got runtime exception when hive syncing hudi_mor_none_part_table_321_0111
    at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:143)
    at org.apache.hudi.sync.common.util.SyncUtilHelpers.runHoodieMetaSync(SyncUtilHelpers.java:59)
    ... 93 more
Caused by: org.apache.hudi.hive.HoodieHiveSyncException: Failed to sync partitions for table hudi_mor_none_part_table_321_0111_ro
    at org.apache.hudi.hive.HiveSyncTool.syncPartitions(HiveSyncTool.java:418)
    at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:232)
    at org.apache.hudi.hive.HiveSyncTool.doSync(HiveSyncTool.java:156)
    at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:140)
    ... 94 more
Caused by: java.lang.IllegalArgumentException: Partition path  is not in the form partition_key=partition_value.
    at org.apache.hudi.hive.HiveStylePartitionValueExtractor.extractPartitionValuesInPath(HiveStylePartitionValueExtractor.java:37)
    at org.apache.hudi.hive.AbstractHiveSyncHoodieClient.getPartitionEvents(AbstractHiveSyncHoodieClient.java:81)
    at org.apache.hudi.hive.HiveSyncTool.syncPartitions(HiveSyncTool.java:396)
    ... 97 more
org.apache.hudi.exception.HoodieException: Could not sync using the meta sync class org.apache.hudi.hive.HiveSyncTool
    at org.apache.hudi.sync.common.util.SyncUtilHelpers.runHoodieMetaSync(SyncUtilHelpers.java:61)
    at org.apache.hudi.HoodieSparkSqlWriter$.$anonfun$metaSync$2(HoodieSparkSqlWriter.scala:626)
    at org.apache.hudi.HoodieSparkSqlWriter$.$anonfun$metaSync$2$adapted(HoodieSparkSqlWriter.scala:625)
    at scala.collection.mutable.HashSet.foreach(HashSet.scala:79)
    at org.apache.hudi.HoodieSparkSqlWriter$.metaSync(HoodieSparkSqlWriter.scala:625)

{code}
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)