You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/05/12 18:57:00 UTC
[jira] [Work logged] (BEAM-14464) Optimize precombining table.
[ https://issues.apache.org/jira/browse/BEAM-14464?focusedWorklogId=769818&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769818 ]
ASF GitHub Bot logged work on BEAM-14464:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 12/May/22 18:56
Start Date: 12/May/22 18:56
Worklog Time Spent: 10m
Work Description: robertwb opened a new pull request, #17641:
URL: https://github.com/apache/beam/pull/17641
The current code constructs, hashes, and compares full WindowedValues
for the grouping key, which ends up dominating the time spent in the
combining table when using trivial combiners (like Sum.integers()).
We only need compare the (structural value of the) key and windows,
and can emit the windows in the global case.
Per added microbenchmarks, this is roughly a 50% improvement for
singly-windowed values, and roughly 2.5x improvement for the common
GlobalWindows case.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
Issue Time Tracking
-------------------
Worklog Id: (was: 769818)
Remaining Estimate: 0h
Time Spent: 10m
> Optimize precombining table.
> ----------------------------
>
> Key: BEAM-14464
> URL: https://issues.apache.org/jira/browse/BEAM-14464
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core
> Reporter: Robert Bradshaw
> Priority: P2
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The current code constructs, hashes, and compares full WindowedValues for the grouping key, which ends up dominating the time spent in the combining table when using trivial combiners (like Sum.integers()). We only need compare the (structural value of the) key and windows, and can emit the windows in the global case.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)