You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Robert Burke (JIRA)" <ji...@apache.org> on 2019/01/24 23:50:00 UTC
[jira] [Assigned] (BEAM-5977) Benchmark buffer use in the Go SDK
Harness
[ https://issues.apache.org/jira/browse/BEAM-5977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Burke reassigned BEAM-5977:
----------------------------------
Assignee: Robert Burke
> Benchmark buffer use in the Go SDK Harness
> ------------------------------------------
>
> Key: BEAM-5977
> URL: https://issues.apache.org/jira/browse/BEAM-5977
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: Robert Burke
> Assignee: Robert Burke
> Priority: Major
> Fix For: Not applicable
>
>
> There's opportunity to reduce CPU & RAM usage with better buffer re-use in [datamgr.go|https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/harness/datamgr.go] WRT both the small element batch buffer and with large elements.
> There should be some benchmarking around both large elements, and smaller elements of varying sizes, (and ideally mixing), and using that we can measure subsequent improvements.
> As a performance improvement, offhand, maintaining a pair of a couple of `chunkSize` buffers could be handy and avoid associated GRPC costs, to handle the smaller elements, as well as flushing large elements immediately and without copying.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)