You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/05/13 22:55:24 UTC
[GitHub] [beam] lostluck opened a new pull request, #17673: [BEAM-14470] Use Generic Registrations in loadtests.
lostluck opened a new pull request, #17673:
URL: https://github.com/apache/beam/pull/17673
This PR uses the new generic registrations to optimize the performance of the load tests WRT reflective invocation. This includes DoFns, Functions, Iterators, and Emitters.
This required adding registrations for the synthetic SDF source, the RuntimeMonitor, and any test specific DoFns or functions.
In many cases, performance has improved! But not quite to Java levels. That JIT is black magic, though Go might be able to get closer with some effort.
In addition:
* Always processes the metrics, and print out the measurement in human readable formats, even if we aren't uploading.
* Allows for a "mixed mode" method approach for the generic method invocation wrappers.
* Previous behavior would cause a break for SDFs and similar, since the methods were unavailable. This allows for a fallback using standard reflective wrappers for the additional low frequency methods.
* Fixes the SideInput load tests: they weren't using the same load as the python tests, nor were they measuring from the correct point (after initializing the side input data).
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on a diff in pull request #17673: [BEAM-14470] Use Generic Registrations in loadtests.
Posted by GitBox <gi...@apache.org>.
lostluck commented on code in PR #17673:
URL: https://github.com/apache/beam/pull/17673#discussion_r873082283
##########
sdks/go/test/load/sideinput/sideinput.go:
##########
@@ -51,11 +54,17 @@ func parseSyntheticConfig() synthetic.SourceConfig {
}
}
+// impToKV just turns an impulse signal into a KV instead of
+// adding a single value input version of RuntimeMonitor
+func impToKV(imp []byte, emit func([]byte, []byte)) {
+ emit(imp, imp)
+}
+
type doFn struct {
Review Comment:
The focus for this one is that it uses an iterator, so calling that one out, so it's easier to add a Dictionary based one later.
##########
sdks/go/test/load/sideinput/sideinput.go:
##########
@@ -51,11 +54,17 @@ func parseSyntheticConfig() synthetic.SourceConfig {
}
}
+// impToKV just turns an impulse signal into a KV instead of
+// adding a single value input version of RuntimeMonitor
+func impToKV(imp []byte, emit func([]byte, []byte)) {
+ emit(imp, imp)
+}
+
type doFn struct {
Review Comment:
The focus for this one is that it uses an iterator, so calling that one out, so it's easier to add a map based one later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck merged pull request #17673: [BEAM-14470] Use Generic Registrations in loadtests.
Posted by GitBox <gi...@apache.org>.
lostluck merged PR #17673:
URL: https://github.com/apache/beam/pull/17673
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #17673: [BEAM-14470] Use Generic Registrations in loadtests.
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on PR #17673:
URL: https://github.com/apache/beam/pull/17673#issuecomment-1126564044
# [Codecov](https://codecov.io/gh/apache/beam/pull/17673?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#17673](https://codecov.io/gh/apache/beam/pull/17673?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8d7f988) into [master](https://codecov.io/gh/apache/beam/commit/2b7aab0ea7236265840f6c3795deb5eea6188ba9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2b7aab0) will **increase** coverage by `0.00%`.
> The diff coverage is `71.42%`.
```diff
@@ Coverage Diff @@
## master #17673 +/- ##
=======================================
Coverage 73.97% 73.97%
=======================================
Files 693 693
Lines 91724 91729 +5
=======================================
+ Hits 67853 67857 +4
Misses 22622 22622
- Partials 1249 1250 +1
```
| Flag | Coverage Δ | |
|---|---|---|
| go | `50.37% <71.42%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/17673?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/go/pkg/beam/core/graph/fn.go](https://codecov.io/gh/apache/beam/pull/17673/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL2dyYXBoL2ZuLmdv) | `76.82% <0.00%> (-0.09%)` | :arrow_down: |
| [sdks/go/pkg/beam/io/synthetic/source.go](https://codecov.io/gh/apache/beam/pull/17673/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9pby9zeW50aGV0aWMvc291cmNlLmdv) | `64.58% <100.00%> (ø)` | |
| [sdks/go/pkg/beam/io/synthetic/step.go](https://codecov.io/gh/apache/beam/pull/17673/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9pby9zeW50aGV0aWMvc3RlcC5nbw==) | `81.81% <100.00%> (+0.27%)` | :arrow_up: |
| [sdks/go/pkg/beam/core/runtime/exec/sdf.go](https://codecov.io/gh/apache/beam/pull/17673/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9jb3JlL3J1bnRpbWUvZXhlYy9zZGYuZ28=) | `71.01% <0.00%> (+0.18%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/17673?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/17673?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2b7aab0...8d7f988](https://codecov.io/gh/apache/beam/pull/17673?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on a diff in pull request #17673: [BEAM-14470] Use Generic Registrations in loadtests.
Posted by GitBox <gi...@apache.org>.
lostluck commented on code in PR #17673:
URL: https://github.com/apache/beam/pull/17673#discussion_r873082268
##########
sdks/go/test/load/util.go:
##########
@@ -132,10 +133,16 @@ func newLoadTestResult(value float64) loadTestResult {
// PublishMetrics calculates the runtime and sends the result to InfluxDB database.
func PublishMetrics(results metrics.QueryResults) {
options := newInfluxDBOptions()
- if options.validate() {
- if res := toLoadTestResults(results); len(res) > 0 {
- publishMetricstoInfluxDB(options, toLoadTestResults(results))
- }
+ ress := toLoadTestResults(results)
+ for _, res := range ress {
+ log.Printf("%s %v", res.metric, time.Duration(float64(time.Second)*res.value))
+ }
+ if len(ress) == 0 {
+ log.Print("No metrics returned.")
+ return
+ }
+ if options.validate() && len(ress) > 0 {
Review Comment:
Nice catch. Nah, no metrics means it doesn't mean anything for the DB options to be correct or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #17673: [BEAM-14470] Use Generic Registrations in loadtests.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #17673:
URL: https://github.com/apache/beam/pull/17673#issuecomment-1126578075
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on pull request #17673: [BEAM-14470] Use Generic Registrations in loadtests.
Posted by GitBox <gi...@apache.org>.
lostluck commented on PR #17673:
URL: https://github.com/apache/beam/pull/17673#issuecomment-1126577732
R: @damccorm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #17673: [BEAM-14470] Use Generic Registrations in loadtests.
Posted by GitBox <gi...@apache.org>.
damccorm commented on code in PR #17673:
URL: https://github.com/apache/beam/pull/17673#discussion_r872912707
##########
sdks/go/test/load/sideinput/sideinput.go:
##########
@@ -51,11 +54,17 @@ func parseSyntheticConfig() synthetic.SourceConfig {
}
}
+// impToKV just turns an impulse signal into a KV instead of
+// adding a single value input version of RuntimeMonitor
+func impToKV(imp []byte, emit func([]byte, []byte)) {
+ emit(imp, imp)
+}
+
type doFn struct {
Review Comment:
I know this predates the change, but would you mind giving this a meaningful name? Maybe something like firstNKVPairs
##########
sdks/go/test/load/util.go:
##########
@@ -132,10 +133,16 @@ func newLoadTestResult(value float64) loadTestResult {
// PublishMetrics calculates the runtime and sends the result to InfluxDB database.
func PublishMetrics(results metrics.QueryResults) {
options := newInfluxDBOptions()
- if options.validate() {
- if res := toLoadTestResults(results); len(res) > 0 {
- publishMetricstoInfluxDB(options, toLoadTestResults(results))
- }
+ ress := toLoadTestResults(results)
+ for _, res := range ress {
+ log.Printf("%s %v", res.metric, time.Duration(float64(time.Second)*res.value))
+ }
+ if len(ress) == 0 {
+ log.Print("No metrics returned.")
+ return
+ }
+ if options.validate() && len(ress) > 0 {
Review Comment:
```suggestion
if options.validate() {
```
We've already early returned in this case so we probably don't need this. If you want to be extra explicit you can put it in the else (I'd just cut the extra clause though)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org