[GitHub] [hudi] nsivabalan commented on pull request #1819: [HUDI-1058] Make delete marker configurable

[GitBox <gi...@apache.org>]

nsivabalan commented on pull request #1819:
URL: https://github.com/apache/hudi/pull/1819#issuecomment-658130555


   Looks like there are 2 options here. 
   Option1: Change interface for combineAndGetUpdateValue and getInsertValue to take in delete field.
   check https://github.com/apache/hudi/pull/1792 on why we need to fix getInsertValue as well. 
   Option2: Approach taken in the patch, where in expose a setDeleteField method in OverwriteWithLatestAvroPayload and this will be set while generating HoodieRecords from GenericRecord.
   
   Since making changes in getInsertValue will touch lot of files, guess we can go with Option2 with some changes. 
   Instead of exposing a new method, why can't we add an overloaded constructor. When HoodieSparkSqlWriter calls into 
   ```
   DataSourceUtils.createHoodieRecord(gr,
                   orderingVal, keyGenerator.getKey(gr), parameters(PAYLOAD_CLASS_OPT_KEY))
   ```
   we could also pass in a delete field optionally which inturn will be passed into constructor of OverwriteWithLatestAvroPayload. If delete field is set by user, then we take that else we resort to "_hoodie_is_deleted".
   
   Let me know your thoughts.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org