You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/12/26 23:41:00 UTC
[jira] [Work logged] (BEAM-4726) Reduce ParDo per element Invoke
overhead
[ https://issues.apache.org/jira/browse/BEAM-4726?focusedWorklogId=178814&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-178814 ]
ASF GitHub Bot logged work on BEAM-4726:
----------------------------------------
Author: ASF GitHub Bot
Created on: 26/Dec/18 23:40
Start Date: 26/Dec/18 23:40
Worklog Time Spent: 10m
Work Description: lostluck commented on pull request #7355: [BEAM-4726] Add arity specialization for calling and returns.
URL: https://github.com/apache/beam/pull/7355
Avoid allocations of return slices when invoking DoFns, and cache the type conversion transforms. Largely lives with the invariant that within a DoFn, all inputs will be of the same type, and avoids the overhead of the type switch in Convert on a perDoFn basis.
This adds more overhead on the initial call for a DoFn, but saves ~30-50ns per invocation avoiding allocation overhead, and type comparison overhead.
------------------------
Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
It will help us expedite review of your Pull Request if you tag someone (e.g. `@username`) to look at it.
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | --- | --- | --- | ---
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/) [](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/) </br> [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/) | --- | --- | ---
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 178814)
Time Spent: 8h (was: 7h 50m)
> Reduce ParDo per element Invoke overhead
> ----------------------------------------
>
> Key: BEAM-4726
> URL: https://issues.apache.org/jira/browse/BEAM-4726
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-go
> Reporter: Robert Burke
> Assignee: Robert Burke
> Priority: Major
> Time Spent: 8h
> Remaining Estimate: 0h
>
> Each call to invoke allocates a new args []interface{}, but the common case is to run the same ProcessElement function over and again. It should be possible to have a container struct to retain the args slice, and avoid recomputing the indices for where to assign parameters before calling the ProcessElementFn.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)