You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Robert Burke (JIRA)" <ji...@apache.org> on 2018/08/06 20:45:00 UTC

[jira] [Commented] (BEAM-4726) Reduce ParDo per element Invoke overhead

    [ https://issues.apache.org/jira/browse/BEAM-4726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16570753#comment-16570753 ] 

Robert Burke commented on BEAM-4726:
------------------------------------

It might be possible to further reduce the hot path per element and remove the conditional checks on inputs, such as extra arguments (like emitters & side inputs), and handling outputs, as are consistent per invocation.

 

Except for error handling, everything is identical between invocations, so it might be possible to target the specific assignments into the args, and avoid repeating values that aren't changing (eg, the context, the emitters & side input iterators, which only need to be re-set.)

> Reduce ParDo per element Invoke overhead
> ----------------------------------------
>
>                 Key: BEAM-4726
>                 URL: https://issues.apache.org/jira/browse/BEAM-4726
>             Project: Beam
>          Issue Type: Sub-task
>          Components: sdk-go
>            Reporter: Robert Burke
>            Assignee: Robert Burke
>            Priority: Major
>          Time Spent: 7h 20m
>  Remaining Estimate: 0h
>
> Each call to invoke allocates a new args []interface{}, but the common case is to run the same ProcessElement function over and again. It should be possible to have a container struct to retain the args slice, and avoid recomputing the indices for where to assign parameters before calling the ProcessElementFn.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)