You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "voonhous (via GitHub)" <gi...@apache.org> on 2026/07/31 13:01:44 UTC

[I] SQL UPDATE on MOR table with hoodie.populate.meta.fields=false writes a log file with an empty fileId [hudi]

voonhous opened a new issue, #19448:
URL: https://github.com/apache/hudi/issues/19448

   ### Problem
   
   A SQL `UPDATE` on a MOR table created with `hoodie.populate.meta.fields = 'false'` fails: the append handle writes a log file whose name has an empty fileId segment, which `HoodieLogFile.parseFieldsFromPath` then rejects.
   
   ### Repro (master, spark3.5 profile; taken verbatim from the failing test, not minimized)
   
   ```sql
   CREATE TABLE t USING HUDI
   PARTITIONED BY (part)
   TBLPROPERTIES (
     type = 'mor',
     primaryKey = 'id',
     orderingFields = 'ts',
     hoodie.populate.meta.fields = 'false'
   )
   LOCATION '/tmp/t'
   AS SELECT
     1 AS id,
     named_struct('name', 'a1', 'price', 10) AS item,
     array(named_struct('k', 'k0', 'v', 'v0')) AS tags,
     map('m0', named_struct('a', 1, 'b', 2)) AS props,
     123456 AS ts,
     'p1' AS part;
   
   UPDATE t SET ts = 123457 WHERE id = 1;
   ```
   
   fails with
   
   ```
   org.apache.hudi.exception.HoodieUpsertException: Error upserting bucketType UPDATE for partition :0
   Caused by: org.apache.hudi.exception.InvalidHoodiePathException: Invalid path
   file:/.../_0-106-150_20260731123923502_1.log.parquet of type LogFile
       at org.apache.hudi.common.model.HoodieLogFile.parseFieldsFromPath(HoodieLogFile.java:129)
   ```
   
   Note the log file name starts with `_` -- the fileId segment is empty. The same statements succeed with meta fields enabled.
   
   ### Expected
   
   Either the update works with virtual keys, or the write fails fast with a clear unsupported-operation error instead of writing an unparseable log file path.
   
   Found while extending `TestNestedSchemaPruningOptimization` in #19406.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org