You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Robert Burke (JIRA)" <ji...@apache.org> on 2018/06/04 18:26:00 UTC

[jira] [Updated] (BEAM-4468) Go SDK-Tune in memory pre-combine caching for Lifted Combines.

     [ https://issues.apache.org/jira/browse/BEAM-4468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Burke updated BEAM-4468:
-------------------------------
    Summary: Go SDK-Tune in memory pre-combine caching for Lifted Combines.  (was: Tune in memory pre-combine caching for Lifted Combines.)

> Go SDK-Tune in memory pre-combine caching for Lifted Combines.
> --------------------------------------------------------------
>
>                 Key: BEAM-4468
>                 URL: https://issues.apache.org/jira/browse/BEAM-4468
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-go
>            Reporter: Robert Burke
>            Assignee: Henning Rohde
>            Priority: Minor
>
> Requires [BEAM-4276|https://issues.apache.org/jira/browse/BEAM-4276] to be completed first.
> Additional performance tweaks to the in memory per-key accumulator cache during the pre-combine phase of a lifted combine.
> This can include any of
>  * capping the number of key-accumulator in the cache, and draining them eagerly after x elements seen, or evicting keys by some heuristic after the cache has seen X distinct keys.
>  * providing a counter on cache size (key/element counts) exposable through the metrics interface or another runner standard counter, to permit observing the cache's status, especially if it could grow without bound within a bundle.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)