[GitHub] [hudi] pankajbsn opened a new issue, #8103: [SUPPORT] How to update record in OverwriteWithLatestAvroPayload.preCombine

["pankajbsn (via GitHub)" <gi...@apache.org>]

pankajbsn opened a new issue, #8103:
URL: https://github.com/apache/hudi/issues/8103

   1. I have a requirement where I need to combine fields from old and new records in OverwriteWithLatestAvroPayload.preCombine
   2. The default behaviour either selects old-record or `this` based on ordering Value.
   3. But in my case I need to combine fields from both old and new in same record before returning value.
   4. To be able to do this, I am trying to first convert `OverwriteWithLatestAvroPayload` to GenericRecord so that I can modify field values.
   5. In below code, if I return `newRec` as it is, it works as expected i.e. first  `preCombine` is called and then `combineAndGetUpdateValue` is called.
   6. But if I return newRec1 i.e. when I convert `GenericRecord` back to `OverwriteWithLatestAvroPayload` it doesn't work. I.e. `preCombine` is called but `combineAndGetUpdateValue` is not called.
   7. So the question is, how can I modify the record in preCombine before returning.
   
   ```java
       public OverwriteWithLatestAvroPayload preCombine(OverwriteWithLatestAvroPayload oldValue) {
           OverwriteWithLatestAvroPayload newRec = super.preCombine(oldValue);
   
           try {
               OverwriteWithLatestAvroPayload newRec1 = new OverwriteWithLatestAvroPayload(HoodieAvroUtils.bytesToAvro(newRec.recordBytes,defaultSchema), newRec.orderingVal);
               System.out.println(HoodieAvroUtils.avroToJson(HoodieAvroUtils.bytesToAvro(newRec.recordBytes,defaultSchema),true));
               return newRec1;
           } catch (IOException e) {
               e.printStackTrace();
               return newRec;
           }
       }
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] pankajbsn commented on issue #8103: [SUPPORT] How to update record in OverwriteWithLatestAvroPayload.preCombine

["pankajbsn (via GitHub)" <gi...@apache.org>]

pankajbsn commented on issue #8103:
URL: https://github.com/apache/hudi/issues/8103#issuecomment-1456241236

   After changing to following line, it worked. Bottomline is make sure the object instance that you are creating is an instance of your payload class and not the generic OverwriteWithLatestAvroPayload.
   
   ```java
   MsgDetailPayload newRec1 = new MsgDetailPayload(HoodieAvroUtils.bytesToAvro(newRec.recordBytes,defaultSchema), newRec.orderingVal);
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] pankajbsn closed issue #8103: [SUPPORT] How to update record in OverwriteWithLatestAvroPayload.preCombine

["pankajbsn (via GitHub)" <gi...@apache.org>]

pankajbsn closed issue #8103: [SUPPORT] How to update record in OverwriteWithLatestAvroPayload.preCombine
URL: https://github.com/apache/hudi/issues/8103


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org