You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "beyond1920 (via GitHub)" <gi...@apache.org> on 2024/03/24 04:12:06 UTC

Re: [PR] [HUDI-6758] Fixing deducing spurious log blocks due to spark retries [hudi]

beyond1920 commented on PR #9611:
URL: https://github.com/apache/hudi/pull/9611#issuecomment-2016687160

   @nsivabalan Good job. We found a minor drawback. 
   
   There are 4 cases that the task retry:
   1. Task is slow, another speculation task is retried
   2. The task failed and retry
   3. The stage failed and retry
   4. The executor failed and retry
   For the third point, which stage retried. Task attempt number might be back to the original value.
   Using attempt number to identify the block is not enough to handle this case. It might leads to wrong result to compare blocks size of each attempt no.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org