You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by "kishorepola (via GitHub)" <gi...@apache.org> on 2026/03/01 08:06:25 UTC

Re: [PR] If the partition count or kafka IO size is large, then skip committin… [beam]

kishorepola commented on PR #37510:
URL: https://github.com/apache/beam/pull/37510#issuecomment-3979457323

   @tomstepp @johnjcasey Thank you for the thorough review! I've addressed all the feedback:
   
   ✅ Refactored `commitCheckpointMark` to use Java streams for better consistency
   ✅ Added debug logging to track idle partition counts
   ✅ Implemented time-based periodic commits (10-minute interval) for idle partitions to support time lag monitoring
   ✅ Added unit test `KafkaUnboundedReaderIdlePartitionTest` to verify the optimization behavior
   
   All changes maintain backward compatibility and follow spotless formatting standards. Ready for another review!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org