You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "nsivabalan (via GitHub)" <gi...@apache.org> on 2026/06/25 03:16:20 UTC

Re: [I] split_reader don't checkpoint before consuming all splits [hudi]

nsivabalan commented on issue #15890:
URL: https://github.com/apache/hudi/issues/15890#issuecomment-4795497642

   This appears to already be fixed on master. The mini-batch consumption fix landed in [HUDI-2485 / PR #3710](https://github.com/apache/hudi/pull/3710) (commit `31a301f0`, Sept 2021). `StreamReadOperator` now reads at most `MINI_BATCH_SIZE = 2048` records per mailbox execution, sets state back to `IDLE`, and re-enqueues via the `MailboxExecutor` — which lets checkpoint barriers interleave between mini-batches rather than waiting for all splits to be consumed. The class-level comment in `StreamReadOperator.java` documents this design intent explicitly. Closing as fixed; please reopen with a concrete repro on current master if you still see the issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org