You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "nsivabalan (via GitHub)" <gi...@apache.org> on 2024/04/09 02:26:37 UTC
Re: [I] [SUPPORT] After enable speculation execution of spark compaction job, some broken parquet files might be generated [hudi]
nsivabalan commented on issue #9615:
URL: https://github.com/apache/hudi/issues/9615#issuecomment-2044040888
We gonna attempt at fixing the issue on this using completion markers. Will post an update shortly on how we plan to tackle this.
But in the mean time, curious to know how you folks are detecting these additional parquet files. There are chances it could lead to duplicates right? how you folks are managing to void data consistency issues?
Until we have a proper fix, trying to gauge if we can suggest some workarounds for other Hudi OSS users.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org