[jira] [Updated] (BEAM-13082) [Go SDK] Reduce churn in dataWriter by retaining byte slice.

["Robert Burke (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Burke updated BEAM-13082:
--------------------------------
    Status: Open  (was: Triage Needed)

> [Go SDK] Reduce churn in dataWriter by retaining byte slice.
> ------------------------------------------------------------
>
>                 Key: BEAM-13082
>                 URL: https://issues.apache.org/jira/browse/BEAM-13082
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-go
>            Reporter: Robert Burke
>            Priority: P2
>
> It's been noted that we can reduce allocations and GC overhead produced by the dataWriter if we change the `w.buf = nil` to `w.buf = w.buf[:0]`. We should still nil out the buffer after the final flush in Close() however, to avoid retaining larger byte buffers after bundle termination.
> A dataWriter is created per bundle, and is only used and is safe to use by that bundle 's processing thread. Further, GRPC's Send call doesn't maintain ownership of the Proto message data after Send returns, allowing this re-use.
> A later optimization could use a sync.Pool to maintain a "freelist" of buffers to further reduce per bundle allocations but this would likely only be noticeable in streaming contexts. Such a free list should have a cap of keeping buffers under some threshold (say slices under 64MB in cap) to avoid retaining overly large buffers that aren't in active use. This idea though is out of scope for a first pass.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)