You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Robert Burke (JIRA)" <ji...@apache.org> on 2018/07/26 17:30:00 UTC

[jira] [Created] (BEAM-5030) Consolidate defer overhead per bundle

Robert Burke created BEAM-5030:
----------------------------------

             Summary: Consolidate defer overhead per bundle
                 Key: BEAM-5030
                 URL: https://issues.apache.org/jira/browse/BEAM-5030
             Project: Beam
          Issue Type: Sub-task
          Components: sdk-go
            Reporter: Robert Burke
            Assignee: Robert Burke


At present, reflectx.CallNoPanic is invoked for every element, and adds  recovery defers for each element. Defers are cheap, but not "per element" cheap. In a simple pipeline (a single ParDo), this represents around ~3% overhead for the pipeline, which is aggravated for every pardo layer.

Since the beam model's unit of re-triable computation is per bundle, the recovery defers should be invoked per bundle only at the root. This is already being done in [exec/plan.go|https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/exec/plan.go#L112]

This could be as simple as calling Call directly in [exec/fn.go|https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/exec/fn.go#L99] instead of with the reflectx wrapper as it is presently.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)