You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Robert Burke (JIRA)" <ji...@apache.org> on 2018/08/06 20:38:00 UTC
[jira] [Assigned] (BEAM-5030) Consolidate defer overhead per bundle
[ https://issues.apache.org/jira/browse/BEAM-5030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Burke reassigned BEAM-5030:
----------------------------------
Assignee: holdenk (was: Robert Burke)
> Consolidate defer overhead per bundle
> -------------------------------------
>
> Key: BEAM-5030
> URL: https://issues.apache.org/jira/browse/BEAM-5030
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-go
> Reporter: Robert Burke
> Assignee: holdenk
> Priority: Major
> Fix For: 2.7.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> At present, reflectx.CallNoPanic is invoked for every element, and adds recovery defers for each element. Defers are cheap, but not "per element" cheap. In a simple pipeline (a single ParDo), this represents around ~3% overhead for the pipeline, which is aggravated for every pardo layer.
> Since the beam model's unit of re-triable computation is per bundle, the recovery defers should be invoked per bundle only at the root. This is already being done in [exec/plan.go|https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/exec/plan.go#L112]
> This could be as simple as calling Call directly in [exec/fn.go|https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/exec/fn.go#L99] instead of with the reflectx wrapper as it is presently.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)