You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "nsivabalan (via GitHub)" <gi...@apache.org> on 2026/06/25 03:00:23 UTC

Re: [I] automatically adapt COMMITS_ARCHIVAL_BATCH_SIZE [hudi]

nsivabalan commented on issue #15473:
URL: https://github.com/apache/hudi/issues/15473#issuecomment-4795432012

   This issue appears to be obsolete on current `master`. The original failure mode — log-block overflow when `COMMITS_ARCHIVAL_BATCH_SIZE` (count-based) batches together several large instants — is specific to the V1 archiver (`TimelineArchiverV1.archive()`), which packs archived entries into a `HoodieAvroDataBlock` log block.
   
   Since [HUDI-6539](https://issues.apache.org/jira/browse/HUDI-6539) (commit d924f181), archived timeline is written by `TimelineArchiverV2` / `LSMTimelineWriter` as Parquet files in an LSM tree (one row per instant, no log blocks). `LAYOUT_VERSION_2` is now the default (`TimelineLayoutVersion.CURR_LAYOUT_VERSION`), and `COMMITS_ARCHIVAL_BATCH_SIZE` is not referenced anywhere in the V2 code path. LSM compaction is already size-bounded via `hoodie.timeline.compaction.target.file.max.bytes` (`LSMTimelineWriter.getCandidateFiles`), which is essentially the size-based gating originally requested.
   
   Proposing to close as fixed by HUDI-6539. If anyone is still running on the legacy V1 layout and hits this, please reopen with details.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org