Re: [I] Different parquet reader config on list-typed fields is used to read parquet file generated by clustering [hudi]

["rangareddy (via GitHub)" <gi...@apache.org>]

rangareddy commented on issue #17281:
URL: https://github.com/apache/hudi/issues/17281#issuecomment-5365812431

   This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-8299).
   
   **Findings: needs a reproduction against a current build before it can be dispositioned.**
   
   The report is concrete and actionable as written: HoodieAvroParquetReader returns null for a list-typed field when reading a parquet file produced by a replacecommit (clustering), causing an NPE during global-index tagging via HoodieMergedReadHandle.
   
   I have deliberately not called this one from the source. The surrounding code has been substantially rewritten since it was filed - table versions 8 and 9, the `BufferedRecordMerger` family replacing payload-based merging, the file-group-reader migration, and the `hudi-common` package reorganisations in #19193 / #19195. With that much churn, inferring either "still broken" or "silently fixed" would be a guess rather than a finding, and both errors are costly here.
   
   The good news is that reproducing it is cheap: the description carries a full stack trace through HoodieRecordUtils.loadPayload and SpillableMapUtils.convertToHoodieRecordPayload, and identifies the divergence as a parquet reader config difference on list types.
   
   There is also no commit anywhere in the tree referencing HUDI-8299, so if this was fixed it was incidental to another change rather than deliberate - which is another reason to verify rather than assume.
   
   Keeping this open.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org