Re: [PR] [HUDI-18383] Support selective meta field population [hudi]

["nsivabalan (via GitHub)" <gi...@apache.org>]

nsivabalan commented on PR #18384:
URL: https://github.com/apache/hudi/pull/18384#issuecomment-4173720033

   I don't think this can be a writer property. 
   In case of MOR table, for snapshot reads, to merge base file and log files, we need to know if `_hoodie_record_key` meta field is populated or not. Thats why we added `hoodie.populate.meta.fields` to table property so that, readers can rely on that. 
   
   And another reason why can't this be just a writer property: 
   we can't let users switch between true and false for this. 
   for eg, in first 5 commits, meta fields were null. and in next 5 commits, if meta fields were enabled, the merge handle will assume that meta fields will be available and will use that to merge w/ previous base file. but in previous base file the meta fields could be empty. 
   
   Lets rethink the solution. I have not reviewing the patch for now. Lets get an alignment on the requirements and approach. 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org