You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by "ManvithPanyam (via GitHub)" <gi...@apache.org> on 2026/10/05 16:03:29 UTC

Re: [I] [Feature Request][Go SDK]: Improve TextIO splitting performance [beam]

ManvithPanyam commented on issue #25892:
URL: https://github.com/apache/beam/issues/25892#issuecomment-5998218371

   Hi @lostluck, I'd like to work on this issue as a Go SDK contribution. I traced the current Go TextIO splitting path on master and confirmed that SplitRestriction uses a fixed 64 MB chunk size, while the Java/Python implementations use the adaptive sqrt-based strategy described here. I'd like to investigate implementing the reusable chunk-sizing logic in the offsetrange restriction layer and update TextIO to use it, along with regression/unit tests. Please let me know if this approach still makes sense for the current Beam 3.0 direction.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org