Re: [PR] fix(spark): preserve delete payloads during record reconstruction [hudi]

["hudi-agent (via GitHub)" <gi...@apache.org>]

hudi-agent commented on code in PR #20341:
URL: https://github.com/apache/hudi/pull/20341#discussion_r4280730722


##########
hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/BaseSparkInternalRecordContext.java:
##########
@@ -124,7 +124,7 @@ public String getMetaFieldValue(InternalRow record, int pos) {
   @Override
   public HoodieRecord<InternalRow> constructHoodieRecord(BufferedRecord<InternalRow> bufferedRecord, String partitionPath) {
     HoodieKey hoodieKey = new HoodieKey(bufferedRecord.getRecordKey(), partitionPath);
-    if (bufferedRecord.isDelete()) {
+    if (bufferedRecord.isDelete() && bufferedRecord.isEmpty()) {

Review Comment:
   🤖 Payload-bearing deletes now reach `HoodieInlineLogAppendHandle.writeDelete()` during log compaction as a `HoodieSparkRecord` whose row includes meta fields. That method calls `getOrderingValueAsJava(writeSchema, ...)` without meta fields, and the lookup is positional, so `ts` would be read from a meta column such as `_hoodie_commit_seqno`. This also affects int/long ordering, which worked before through `HoodieEmptyRecord`, and the result is a wrong ordering value written into the compacted delete block rather than an error. Could you have `writeDelete` use its `schema` argument, which is `writeSchemaWithMetaFields` when `useWriterSchema` is set, and add a log-compaction case to the test?
   ```java
   final Comparable<?> orderingVal = hoodieRecord.getOrderingValueAsJava(schema, recordProperties, orderingFields);
   ```
   
   <sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org