You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2021/02/23 21:34:06 UTC
[GitHub] [beam] pabloem opened a new pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
pabloem opened a new pull request #14046:
URL: https://github.com/apache/beam/pull/14046
Adding a microbenchmark for running a simple pipeline with TestStream and a few groupings in parallel.
r: @rohdesamuel
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2
--- | --- | --- | --- | --- | --- | ---
Go | [](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | ---
Java | [](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://ci-beam
.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/)
Python | [](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/) | ---
XLang | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/) | ---
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website | Whitespace | Typescript
--- | --- | --- | --- | --- | --- | ---
Non-portable | [](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/) <br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/)
Portable | --- | [](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | --- | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784541216
# [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=h1) Report
> Merging [#14046](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=desc) (5f8a15f) into [master](https://codecov.io/gh/apache/beam/commit/aaad864c9acb22e35050f974a7ac74fb7638f085?el=desc) (aaad864) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #14046 +/- ##
==========================================
- Coverage 82.98% 82.97% -0.01%
==========================================
Files 469 469
Lines 58298 58300 +2
==========================================
- Hits 48379 48375 -4
- Misses 9919 9925 +6
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...amples/snippets/transforms/elementwise/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9lbGVtZW50d2lzZS9fX2luaXRfXy5weQ==) | | |
| [sdks/python/apache\_beam/examples/snippets/util.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdXRpbC5weQ==) | | |
| [sdks/python/apache\_beam/io/filebasedsource.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmlsZWJhc2Vkc291cmNlLnB5) | | |
| [...am/portability/api/standard\_window\_fns\_pb2\_urns.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL3N0YW5kYXJkX3dpbmRvd19mbnNfcGIyX3VybnMucHk=) | | |
| [...beam/testing/load\_tests/load\_test\_metrics\_utils.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9sb2FkX3Rlc3RzL2xvYWRfdGVzdF9tZXRyaWNzX3V0aWxzLnB5) | | |
| [...he\_beam/portability/api/external\_transforms\_pb2.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2V4dGVybmFsX3RyYW5zZm9ybXNfcGIyLnB5) | | |
| [sdks/python/apache\_beam/coders/typecoders.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL3R5cGVjb2RlcnMucHk=) | | |
| [sdks/python/apache\_beam/examples/sql\_taxi.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc3FsX3RheGkucHk=) | | |
| [...ks/python/apache\_beam/runners/worker/statecache.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc3RhdGVjYWNoZS5weQ==) | | |
| [...\_beam/testing/benchmarks/nexmark/queries/query0.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTAucHk=) | | |
| ... and [928 more](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=footer). Last update [aaad864...5f8a15f](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] rohdesamuel commented on a change in pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
rohdesamuel commented on a change in pull request #14046:
URL: https://github.com/apache/beam/pull/14046#discussion_r582208099
##########
File path: sdks/python/apache_beam/tools/teststream_microbenchmark.py
##########
@@ -0,0 +1,129 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""A microbenchmark for measuring changes in the performance of TestStream
+running locally.
+This microbenchmark attempts to measure the overhead of the main data paths
+for the TestStream. Specifically new elements, watermark changes and processing
+time advances.
+
+This runs a series of N parallel pipelines with M parallel stages each. Each
+stage does the following:
+
+1) Put all the PCollection elements in state
+2) Set a timer for the future
+3) When the timer fires, change the key and output all the elements downstream
+
+This executes the same codepaths that are run on the Fn API (and Dataflow)
+workers, but is generally easier to run (locally) and more stable..
+
+Run as
+
+ python -m apache_beam.tools.teststream_microbenchmark
+
+"""
+
+# pytype: skip-file
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import argparse
+import itertools
+import logging
+import random
+from builtins import range
+
+import apache_beam as beam
+import apache_beam.typehints.typehints as typehints
+from apache_beam import WindowInto
+from apache_beam.runners import DirectRunner
+from apache_beam.testing.test_stream import TestStream
+from apache_beam.tools import utils
+from apache_beam.transforms.window import FixedWindows
+
+NUM_PARALLEL_STAGES = 7
+
+NUM_SERIAL_STAGES = 6
+
+
+class RekeyElements(beam.DoFn):
+ def process(self, element):
+ _, values = element
+ return [(random.randint(0, 1000), v) for v in values]
+
+
+def _build_serial_stages(input_pc, num_serial_stages, stage_count):
+ pc = (input_pc | ('gbk_start_stage%s' % stage_count) >> beam.GroupByKey())
+
+ for i in range(num_serial_stages):
+ pc = (
+ pc
+ | ('stage%s_map%s' % (stage_count, i)) >> beam.ParDo(
+ RekeyElements()).with_output_types(typehints.KV[int, int])
+ | ('stage%s_gbk%s' % (stage_count, i)) >> beam.GroupByKey())
+
+ return pc
+
+
+def run_single_pipeline(size):
+ def _pipeline_runner():
+ with beam.Pipeline(runner=DirectRunner()) as p:
+ ts = TestStream().advance_watermark_to(0)
+ all_elements = iter(range(size))
+ watermark = 0
+ while True:
+ next_batch = list(itertools.islice(all_elements, 100))
+ if not next_batch:
+ break
+ ts = ts.add_elements([(i, random.randint(0, 1000)) for i in next_batch])
+ watermark = watermark + 100
+ ts = ts.advance_watermark_to(watermark)
+ ts = ts.advance_watermark_to_infinity()
+
+ input_pc = p | ts | WindowInto(FixedWindows(100))
+ for i in range(NUM_PARALLEL_STAGES):
+ _build_serial_stages(input_pc, NUM_SERIAL_STAGES, i)
+
+ return _pipeline_runner
+
+
+def run_benchmark(
+ starting_point=1, num_runs=10, num_elements_step=300, verbose=True):
+ suite = [
+ utils.LinearRegressionBenchmarkConfig(
Review comment:
I'm confused, why is this running a linear regression? Shouldn't it just be a windowed average over the last N runs?
##########
File path: sdks/python/apache_beam/tools/teststream_microbenchmark.py
##########
@@ -0,0 +1,129 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""A microbenchmark for measuring changes in the performance of TestStream
+running locally.
+This microbenchmark attempts to measure the overhead of the main data paths
+for the TestStream. Specifically new elements, watermark changes and processing
+time advances.
+
+This runs a series of N parallel pipelines with M parallel stages each. Each
+stage does the following:
+
+1) Put all the PCollection elements in state
+2) Set a timer for the future
+3) When the timer fires, change the key and output all the elements downstream
Review comment:
This comment is somewhat misleading. Technically, windows are state & timers but I don't know if many people will make that connection reading this code. Can you please rewrite what this microbenchmark does in terms of windowing?
##########
File path: sdks/python/apache_beam/tools/teststream_microbenchmark.py
##########
@@ -0,0 +1,129 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""A microbenchmark for measuring changes in the performance of TestStream
+running locally.
+This microbenchmark attempts to measure the overhead of the main data paths
+for the TestStream. Specifically new elements, watermark changes and processing
+time advances.
+
+This runs a series of N parallel pipelines with M parallel stages each. Each
+stage does the following:
+
+1) Put all the PCollection elements in state
+2) Set a timer for the future
+3) When the timer fires, change the key and output all the elements downstream
+
+This executes the same codepaths that are run on the Fn API (and Dataflow)
+workers, but is generally easier to run (locally) and more stable..
Review comment:
Remove extraneous period
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784541216
# [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=h1) Report
> Merging [#14046](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=desc) (5f8a15f) into [master](https://codecov.io/gh/apache/beam/commit/aaad864c9acb22e35050f974a7ac74fb7638f085?el=desc) (aaad864) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #14046 +/- ##
==========================================
- Coverage 82.98% 82.97% -0.01%
==========================================
Files 469 469
Lines 58298 58300 +2
==========================================
- Hits 48379 48375 -4
- Misses 9919 9925 +6
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/internal/pickler.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvcGlja2xlci5weQ==) | | |
| [...\_beam/io/gcp/datastore/v1new/adaptive\_throttler.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2RhdGFzdG9yZS92MW5ldy9hZGFwdGl2ZV90aHJvdHRsZXIucHk=) | | |
| [...examples/snippets/transforms/aggregation/sample.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9zYW1wbGUucHk=) | | |
| [...\_beam/testing/benchmarks/nexmark/queries/query8.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTgucHk=) | | |
| [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | | |
| [sdks/python/apache\_beam/testing/util.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy91dGlsLnB5) | | |
| [...beam/testing/benchmarks/nexmark/queries/query12.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTEyLnB5) | | |
| [...am/testing/benchmarks/chicago\_taxi/trainer/taxi.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL2NoaWNhZ29fdGF4aS90cmFpbmVyL3RheGkucHk=) | | |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | | |
| [.../runners/portability/fn\_api\_runner/translations.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL3RyYW5zbGF0aW9ucy5weQ==) | | |
| ... and [928 more](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=footer). Last update [aaad864...5f8a15f](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on a change in pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
pabloem commented on a change in pull request #14046:
URL: https://github.com/apache/beam/pull/14046#discussion_r582220455
##########
File path: sdks/python/apache_beam/tools/teststream_microbenchmark.py
##########
@@ -0,0 +1,129 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""A microbenchmark for measuring changes in the performance of TestStream
+running locally.
+This microbenchmark attempts to measure the overhead of the main data paths
+for the TestStream. Specifically new elements, watermark changes and processing
+time advances.
+
+This runs a series of N parallel pipelines with M parallel stages each. Each
+stage does the following:
+
+1) Put all the PCollection elements in state
+2) Set a timer for the future
+3) When the timer fires, change the key and output all the elements downstream
Review comment:
Thanks. Great catch. I've rephrased this. LMK if htat looks better.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on a change in pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
pabloem commented on a change in pull request #14046:
URL: https://github.com/apache/beam/pull/14046#discussion_r582221371
##########
File path: sdks/python/apache_beam/tools/teststream_microbenchmark.py
##########
@@ -0,0 +1,129 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""A microbenchmark for measuring changes in the performance of TestStream
+running locally.
+This microbenchmark attempts to measure the overhead of the main data paths
+for the TestStream. Specifically new elements, watermark changes and processing
+time advances.
+
+This runs a series of N parallel pipelines with M parallel stages each. Each
+stage does the following:
+
+1) Put all the PCollection elements in state
+2) Set a timer for the future
+3) When the timer fires, change the key and output all the elements downstream
+
+This executes the same codepaths that are run on the Fn API (and Dataflow)
+workers, but is generally easier to run (locally) and more stable..
+
+Run as
+
+ python -m apache_beam.tools.teststream_microbenchmark
+
+"""
+
+# pytype: skip-file
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import argparse
+import itertools
+import logging
+import random
+from builtins import range
+
+import apache_beam as beam
+import apache_beam.typehints.typehints as typehints
+from apache_beam import WindowInto
+from apache_beam.runners import DirectRunner
+from apache_beam.testing.test_stream import TestStream
+from apache_beam.tools import utils
+from apache_beam.transforms.window import FixedWindows
+
+NUM_PARALLEL_STAGES = 7
+
+NUM_SERIAL_STAGES = 6
+
+
+class RekeyElements(beam.DoFn):
+ def process(self, element):
+ _, values = element
+ return [(random.randint(0, 1000), v) for v in values]
+
+
+def _build_serial_stages(input_pc, num_serial_stages, stage_count):
+ pc = (input_pc | ('gbk_start_stage%s' % stage_count) >> beam.GroupByKey())
+
+ for i in range(num_serial_stages):
+ pc = (
+ pc
+ | ('stage%s_map%s' % (stage_count, i)) >> beam.ParDo(
+ RekeyElements()).with_output_types(typehints.KV[int, int])
+ | ('stage%s_gbk%s' % (stage_count, i)) >> beam.GroupByKey())
+
+ return pc
+
+
+def run_single_pipeline(size):
+ def _pipeline_runner():
+ with beam.Pipeline(runner=DirectRunner()) as p:
+ ts = TestStream().advance_watermark_to(0)
+ all_elements = iter(range(size))
+ watermark = 0
+ while True:
+ next_batch = list(itertools.islice(all_elements, 100))
+ if not next_batch:
+ break
+ ts = ts.add_elements([(i, random.randint(0, 1000)) for i in next_batch])
+ watermark = watermark + 100
+ ts = ts.advance_watermark_to(watermark)
+ ts = ts.advance_watermark_to_infinity()
+
+ input_pc = p | ts | WindowInto(FixedWindows(100))
+ for i in range(NUM_PARALLEL_STAGES):
+ _build_serial_stages(input_pc, NUM_SERIAL_STAGES, i)
+
+ return _pipeline_runner
+
+
+def run_benchmark(
+ starting_point=1, num_runs=10, num_elements_step=300, verbose=True):
+ suite = [
+ utils.LinearRegressionBenchmarkConfig(
Review comment:
This runs a pipeline with 1 element, then 300, then 600, and so on until 3000. We use the linear regression to measure per-element overhead, and base overhead of the runner, and report it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784541216
# [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=h1) Report
> Merging [#14046](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=desc) (7a6b6bb) into [master](https://codecov.io/gh/apache/beam/commit/aaad864c9acb22e35050f974a7ac74fb7638f085?el=desc) (aaad864) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #14046 +/- ##
==========================================
- Coverage 82.98% 82.97% -0.01%
==========================================
Files 469 938 +469
Lines 58298 116600 +58302
==========================================
+ Hits 48379 96751 +48372
- Misses 9919 19849 +9930
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | | |
| [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | | |
| [...amples/snippets/transforms/aggregation/distinct.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9kaXN0aW5jdC5weQ==) | | |
| [...beam/testing/benchmarks/nexmark/queries/query11.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTExLnB5) | | |
| [...ython/apache\_beam/portability/api/endpoints\_pb2.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2VuZHBvaW50c19wYjIucHk=) | | |
| [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | | |
| [...eam/portability/api/beam\_provision\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fcHJvdmlzaW9uX2FwaV9wYjJfZ3JwYy5weQ==) | | |
| [sdks/python/apache\_beam/utils/retry.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvcmV0cnkucHk=) | | |
| [...dks/python/apache\_beam/transforms/external\_java.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbF9qYXZhLnB5) | | |
| [...on/apache\_beam/runners/worker/statesampler\_slow.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc3RhdGVzYW1wbGVyX3Nsb3cucHk=) | | |
| ... and [1397 more](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=footer). Last update [aaad864...7a6b6bb](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784541216
# [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=h1) Report
> Merging [#14046](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=desc) (353e5db) into [master](https://codecov.io/gh/apache/beam/commit/aaad864c9acb22e35050f974a7ac74fb7638f085?el=desc) (aaad864) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #14046 +/- ##
==========================================
- Coverage 82.98% 82.97% -0.01%
==========================================
Files 469 469
Lines 58298 58300 +2
==========================================
- Hits 48379 48376 -3
- Misses 9919 9924 +5
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...on/apache\_beam/runners/direct/helper\_transforms.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvaGVscGVyX3RyYW5zZm9ybXMucHk=) | | |
| [...apache\_beam/runners/portability/portable\_runner.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9wb3J0YWJsZV9ydW5uZXIucHk=) | | |
| [sdks/python/apache\_beam/utils/annotations.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvYW5ub3RhdGlvbnMucHk=) | | |
| [sdks/python/apache\_beam/coders/coders.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL2NvZGVycy5weQ==) | | |
| [...cp/internal/clients/bigquery/bigquery\_v2\_client.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2ludGVybmFsL2NsaWVudHMvYmlncXVlcnkvYmlncXVlcnlfdjJfY2xpZW50LnB5) | | |
| [...hon/apache\_beam/portability/api/schema\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL3NjaGVtYV9wYjJfZ3JwYy5weQ==) | | |
| [...\_beam/testing/benchmarks/nexmark/queries/query3.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTMucHk=) | | |
| [.../python/apache\_beam/typehints/trivial\_inference.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL3RyaXZpYWxfaW5mZXJlbmNlLnB5) | | |
| [sdks/python/apache\_beam/io/filebasedsource.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmlsZWJhc2Vkc291cmNlLnB5) | | |
| [sdks/python/apache\_beam/io/tfrecordio.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vdGZyZWNvcmRpby5weQ==) | | |
| ... and [928 more](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=footer). Last update [aaad864...5f8a15f](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784541216
# [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=h1) Report
> Merging [#14046](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=desc) (7a6b6bb) into [master](https://codecov.io/gh/apache/beam/commit/aaad864c9acb22e35050f974a7ac74fb7638f085?el=desc) (aaad864) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #14046 +/- ##
==========================================
- Coverage 82.98% 82.97% -0.01%
==========================================
Files 469 469
Lines 58298 58300 +2
==========================================
- Hits 48379 48376 -3
- Misses 9919 9924 +5
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...pache\_beam/runners/portability/portable\_metrics.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9wb3J0YWJsZV9tZXRyaWNzLnB5) | | |
| [...ache\_beam/portability/api/beam\_job\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fam9iX2FwaV9wYjJfZ3JwYy5weQ==) | | |
| [sdks/python/apache\_beam/io/gcp/pubsub.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL3B1YnN1Yi5weQ==) | | |
| [...s/python/apache\_beam/ml/gcp/videointelligenceml.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWwvZ2NwL3ZpZGVvaW50ZWxsaWdlbmNlbWwucHk=) | | |
| [.../python/apache\_beam/runners/worker/statesampler.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc3RhdGVzYW1wbGVyLnB5) | | |
| [sdks/python/apache\_beam/examples/complete/tfidf.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvdGZpZGYucHk=) | | |
| [sdks/python/apache\_beam/utils/profiler.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvcHJvZmlsZXIucHk=) | | |
| [sdks/python/apache\_beam/testing/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9fX2luaXRfXy5weQ==) | | |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | | |
| [...e\_beam/portability/api/beam\_interactive\_api\_pb2.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1faW50ZXJhY3RpdmVfYXBpX3BiMi5weQ==) | | |
| ... and [928 more](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=footer). Last update [aaad864...7a6b6bb](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784541216
# [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=h1) Report
> Merging [#14046](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=desc) (5f8a15f) into [master](https://codecov.io/gh/apache/beam/commit/aaad864c9acb22e35050f974a7ac74fb7638f085?el=desc) (aaad864) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #14046 +/- ##
==========================================
- Coverage 82.98% 82.97% -0.01%
==========================================
Files 469 469
Lines 58298 58300 +2
==========================================
- Hits 48379 48375 -4
- Misses 9919 9925 +6
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [.../apache\_beam/runners/dataflow/internal/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9fX2luaXRfXy5weQ==) | | |
| [sdks/python/apache\_beam/io/jdbc.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vamRiYy5weQ==) | | |
| [...gcp/datastore/v1new/datastore\_write\_it\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2RhdGFzdG9yZS92MW5ldy9kYXRhc3RvcmVfd3JpdGVfaXRfcGlwZWxpbmUucHk=) | | |
| [...ks/python/apache\_beam/runners/worker/opcounters.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvb3Bjb3VudGVycy5weQ==) | | |
| [...n/apache\_beam/examples/complete/game/user\_score.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvZ2FtZS91c2VyX3Njb3JlLnB5) | | |
| [sdks/python/apache\_beam/transforms/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9fX2luaXRfXy5weQ==) | | |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | | |
| [...ache\_beam/io/gcp/datastore/v1new/query\_splitter.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2RhdGFzdG9yZS92MW5ldy9xdWVyeV9zcGxpdHRlci5weQ==) | | |
| [...ache\_beam/runners/portability/expansion\_service.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9leHBhbnNpb25fc2VydmljZS5weQ==) | | |
| [sdks/python/apache\_beam/io/gcp/bigquery.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5LnB5) | | |
| ... and [928 more](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=footer). Last update [aaad864...5f8a15f](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784541216
# [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=h1) Report
> Merging [#14046](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=desc) (7a6b6bb) into [master](https://codecov.io/gh/apache/beam/commit/aaad864c9acb22e35050f974a7ac74fb7638f085?el=desc) (aaad864) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #14046 +/- ##
==========================================
- Coverage 82.98% 82.97% -0.01%
==========================================
Files 469 469
Lines 58298 58300 +2
==========================================
- Hits 48379 48376 -3
- Misses 9919 9924 +5
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...pache\_beam/runners/portability/portable\_metrics.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9wb3J0YWJsZV9tZXRyaWNzLnB5) | | |
| [...ache\_beam/portability/api/beam\_job\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fam9iX2FwaV9wYjJfZ3JwYy5weQ==) | | |
| [sdks/python/apache\_beam/io/gcp/pubsub.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL3B1YnN1Yi5weQ==) | | |
| [...s/python/apache\_beam/ml/gcp/videointelligenceml.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWwvZ2NwL3ZpZGVvaW50ZWxsaWdlbmNlbWwucHk=) | | |
| [.../python/apache\_beam/runners/worker/statesampler.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc3RhdGVzYW1wbGVyLnB5) | | |
| [sdks/python/apache\_beam/examples/complete/tfidf.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvdGZpZGYucHk=) | | |
| [sdks/python/apache\_beam/utils/profiler.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvcHJvZmlsZXIucHk=) | | |
| [sdks/python/apache\_beam/testing/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9fX2luaXRfXy5weQ==) | | |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | | |
| [...e\_beam/portability/api/beam\_interactive\_api\_pb2.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1faW50ZXJhY3RpdmVfYXBpX3BiMi5weQ==) | | |
| ... and [928 more](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=footer). Last update [aaad864...7a6b6bb](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784541216
# [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=h1) Report
> Merging [#14046](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=desc) (5f8a15f) into [master](https://codecov.io/gh/apache/beam/commit/aaad864c9acb22e35050f974a7ac74fb7638f085?el=desc) (aaad864) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #14046 +/- ##
==========================================
- Coverage 82.98% 82.97% -0.01%
==========================================
Files 469 469
Lines 58298 58300 +2
==========================================
- Hits 48379 48375 -4
- Misses 9919 9925 +6
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...on/apache\_beam/portability/api/beam\_job\_api\_pb2.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fam9iX2FwaV9wYjIucHk=) | | |
| [sdks/python/apache\_beam/testing/test\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3BpcGVsaW5lLnB5) | | |
| [...on/apache\_beam/runners/portability/spark\_runner.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9zcGFya19ydW5uZXIucHk=) | | |
| [...s/python/apache\_beam/runners/worker/log\_handler.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvbG9nX2hhbmRsZXIucHk=) | | |
| [...hon/apache\_beam/runners/worker/worker\_pool\_main.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvd29ya2VyX3Bvb2xfbWFpbi5weQ==) | | |
| [...apache\_beam/portability/api/beam\_runner\_api\_pb2.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fcnVubmVyX2FwaV9wYjIucHk=) | | |
| [...m/examples/snippets/transforms/elementwise/keys.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9lbGVtZW50d2lzZS9rZXlzLnB5) | | |
| [sdks/python/apache\_beam/io/range\_trackers.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vcmFuZ2VfdHJhY2tlcnMucHk=) | | |
| [...s/python/apache\_beam/examples/complete/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvX19pbml0X18ucHk=) | | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_perf.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19wZXJmLnB5) | | |
| ... and [928 more](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=footer). Last update [aaad864...5f8a15f](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784541216
# [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=h1) Report
> Merging [#14046](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=desc) (7a6b6bb) into [master](https://codecov.io/gh/apache/beam/commit/aaad864c9acb22e35050f974a7ac74fb7638f085?el=desc) (aaad864) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #14046 +/- ##
==========================================
- Coverage 82.98% 82.97% -0.01%
==========================================
Files 469 469
Lines 58298 58300 +2
==========================================
- Hits 48379 48376 -3
- Misses 9919 9924 +5
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...e\_beam/portability/api/beam\_interactive\_api\_pb2.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1faW50ZXJhY3RpdmVfYXBpX3BiMi5weQ==) | | |
| [sdks/python/apache\_beam/dataframe/partitionings.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL3BhcnRpdGlvbmluZ3MucHk=) | | |
| [.../apache\_beam/options/pipeline\_options\_validator.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vb3B0aW9ucy9waXBlbGluZV9vcHRpb25zX3ZhbGlkYXRvci5weQ==) | | |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | | |
| [...che\_beam/runners/interactive/interactive\_runner.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9ydW5uZXIucHk=) | | |
| [...dks/python/apache\_beam/options/pipeline\_options.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vb3B0aW9ucy9waXBlbGluZV9vcHRpb25zLnB5) | | |
| [...n/apache\_beam/runners/direct/test\_direct\_runner.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9kaXJlY3RfcnVubmVyLnB5) | | |
| [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | | |
| [...he\_beam/examples/cookbook/multiple\_output\_pardo.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29va2Jvb2svbXVsdGlwbGVfb3V0cHV0X3BhcmRvLnB5) | | |
| [...thon/apache\_beam/runners/worker/operation\_specs.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvb3BlcmF0aW9uX3NwZWNzLnB5) | | |
| ... and [928 more](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=footer). Last update [aaad864...7a6b6bb](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784541216
# [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=h1) Report
> Merging [#14046](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=desc) (7a6b6bb) into [master](https://codecov.io/gh/apache/beam/commit/aaad864c9acb22e35050f974a7ac74fb7638f085?el=desc) (aaad864) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #14046 +/- ##
==========================================
- Coverage 82.98% 82.97% -0.01%
==========================================
Files 469 469
Lines 58298 58300 +2
==========================================
- Hits 48379 48376 -3
- Misses 9919 9924 +5
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vX19pbml0X18ucHk=) | | |
| [sdks/python/apache\_beam/utils/windowed\_value.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvd2luZG93ZWRfdmFsdWUucHk=) | | |
| [...ks/python/apache\_beam/examples/cookbook/filters.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29va2Jvb2svZmlsdGVycy5weQ==) | | |
| [...eam/runners/interactive/options/capture\_control.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9vcHRpb25zL2NhcHR1cmVfY29udHJvbC5weQ==) | | |
| [...he\_beam/io/flink/flink\_streaming\_impulse\_source.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmxpbmsvZmxpbmtfc3RyZWFtaW5nX2ltcHVsc2Vfc291cmNlLnB5) | | |
| [sdks/python/apache\_beam/coders/row\_coder.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL3Jvd19jb2Rlci5weQ==) | | |
| [...ks/python/apache\_beam/runners/worker/opcounters.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvb3Bjb3VudGVycy5weQ==) | | |
| [...apache\_beam/examples/complete/juliaset/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvanVsaWFzZXQvX19pbml0X18ucHk=) | | |
| [...s/python/apache\_beam/examples/cookbook/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29va2Jvb2svX19pbml0X18ucHk=) | | |
| [...ython/apache\_beam/io/gcp/datastore/v1new/helper.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2RhdGFzdG9yZS92MW5ldy9oZWxwZXIucHk=) | | |
| ... and [928 more](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=footer). Last update [aaad864...7a6b6bb](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784537518
Run Python Load Tests FnApiRunner Microbenchmark
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on a change in pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
pabloem commented on a change in pull request #14046:
URL: https://github.com/apache/beam/pull/14046#discussion_r582220228
##########
File path: sdks/python/apache_beam/tools/teststream_microbenchmark.py
##########
@@ -0,0 +1,129 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""A microbenchmark for measuring changes in the performance of TestStream
+running locally.
+This microbenchmark attempts to measure the overhead of the main data paths
+for the TestStream. Specifically new elements, watermark changes and processing
+time advances.
+
+This runs a series of N parallel pipelines with M parallel stages each. Each
+stage does the following:
+
+1) Put all the PCollection elements in state
+2) Set a timer for the future
+3) When the timer fires, change the key and output all the elements downstream
+
+This executes the same codepaths that are run on the Fn API (and Dataflow)
+workers, but is generally easier to run (locally) and more stable..
Review comment:
done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] rohdesamuel commented on a change in pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
rohdesamuel commented on a change in pull request #14046:
URL: https://github.com/apache/beam/pull/14046#discussion_r582235224
##########
File path: sdks/python/apache_beam/tools/teststream_microbenchmark.py
##########
@@ -0,0 +1,129 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""A microbenchmark for measuring changes in the performance of TestStream
+running locally.
+This microbenchmark attempts to measure the overhead of the main data paths
+for the TestStream. Specifically new elements, watermark changes and processing
+time advances.
+
+This runs a series of N parallel pipelines with M parallel stages each. Each
+stage does the following:
+
+1) Put all the PCollection elements in state
+2) Set a timer for the future
+3) When the timer fires, change the key and output all the elements downstream
+
+This executes the same codepaths that are run on the Fn API (and Dataflow)
+workers, but is generally easier to run (locally) and more stable..
+
+Run as
+
+ python -m apache_beam.tools.teststream_microbenchmark
+
+"""
+
+# pytype: skip-file
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import argparse
+import itertools
+import logging
+import random
+from builtins import range
+
+import apache_beam as beam
+import apache_beam.typehints.typehints as typehints
+from apache_beam import WindowInto
+from apache_beam.runners import DirectRunner
+from apache_beam.testing.test_stream import TestStream
+from apache_beam.tools import utils
+from apache_beam.transforms.window import FixedWindows
+
+NUM_PARALLEL_STAGES = 7
+
+NUM_SERIAL_STAGES = 6
+
+
+class RekeyElements(beam.DoFn):
+ def process(self, element):
+ _, values = element
+ return [(random.randint(0, 1000), v) for v in values]
+
+
+def _build_serial_stages(input_pc, num_serial_stages, stage_count):
+ pc = (input_pc | ('gbk_start_stage%s' % stage_count) >> beam.GroupByKey())
+
+ for i in range(num_serial_stages):
+ pc = (
+ pc
+ | ('stage%s_map%s' % (stage_count, i)) >> beam.ParDo(
+ RekeyElements()).with_output_types(typehints.KV[int, int])
+ | ('stage%s_gbk%s' % (stage_count, i)) >> beam.GroupByKey())
+
+ return pc
+
+
+def run_single_pipeline(size):
+ def _pipeline_runner():
+ with beam.Pipeline(runner=DirectRunner()) as p:
+ ts = TestStream().advance_watermark_to(0)
+ all_elements = iter(range(size))
+ watermark = 0
+ while True:
+ next_batch = list(itertools.islice(all_elements, 100))
+ if not next_batch:
+ break
+ ts = ts.add_elements([(i, random.randint(0, 1000)) for i in next_batch])
+ watermark = watermark + 100
+ ts = ts.advance_watermark_to(watermark)
+ ts = ts.advance_watermark_to_infinity()
+
+ input_pc = p | ts | WindowInto(FixedWindows(100))
+ for i in range(NUM_PARALLEL_STAGES):
+ _build_serial_stages(input_pc, NUM_SERIAL_STAGES, i)
+
+ return _pipeline_runner
+
+
+def run_benchmark(
+ starting_point=1, num_runs=10, num_elements_step=300, verbose=True):
+ suite = [
+ utils.LinearRegressionBenchmarkConfig(
Review comment:
Still feels very weird to me to be using a regression. Usually those are meant to build models/feature extraction, but this seems the benchmark only needs to report some windowed average. Anyway, that's a conversation for another time.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784541216
# [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=h1) Report
> Merging [#14046](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=desc) (7a6b6bb) into [master](https://codecov.io/gh/apache/beam/commit/aaad864c9acb22e35050f974a7ac74fb7638f085?el=desc) (aaad864) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #14046 +/- ##
==========================================
- Coverage 82.98% 82.97% -0.01%
==========================================
Files 469 938 +469
Lines 58298 116600 +58302
==========================================
+ Hits 48379 96751 +48372
- Misses 9919 19849 +9930
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...on/apache\_beam/runners/direct/helper\_transforms.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvaGVscGVyX3RyYW5zZm9ybXMucHk=) | | |
| [...apache\_beam/runners/portability/portable\_runner.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9wb3J0YWJsZV9ydW5uZXIucHk=) | | |
| [sdks/python/apache\_beam/utils/annotations.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvYW5ub3RhdGlvbnMucHk=) | | |
| [sdks/python/apache\_beam/coders/coders.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL2NvZGVycy5weQ==) | | |
| [...cp/internal/clients/bigquery/bigquery\_v2\_client.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2ludGVybmFsL2NsaWVudHMvYmlncXVlcnkvYmlncXVlcnlfdjJfY2xpZW50LnB5) | | |
| [...hon/apache\_beam/portability/api/schema\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL3NjaGVtYV9wYjJfZ3JwYy5weQ==) | | |
| [...\_beam/testing/benchmarks/nexmark/queries/query3.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTMucHk=) | | |
| [.../python/apache\_beam/typehints/trivial\_inference.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL3RyaXZpYWxfaW5mZXJlbmNlLnB5) | | |
| [sdks/python/apache\_beam/io/filebasedsource.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmlsZWJhc2Vkc291cmNlLnB5) | | |
| [sdks/python/apache\_beam/io/tfrecordio.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vdGZyZWNvcmRpby5weQ==) | | |
| ... and [1397 more](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=footer). Last update [aaad864...353e5db](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784555833
Run Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784549362
Successful run: https://ci-beam.apache.org/job/beam_Python_LoadTests_FnApiRunner_Microbenchmark_PR/5/console
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem merged pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
pabloem merged pull request #14046:
URL: https://github.com/apache/beam/pull/14046
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784541216
# [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=h1) Report
> Merging [#14046](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=desc) (5f8a15f) into [master](https://codecov.io/gh/apache/beam/commit/aaad864c9acb22e35050f974a7ac74fb7638f085?el=desc) (aaad864) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #14046 +/- ##
==========================================
- Coverage 82.98% 82.97% -0.01%
==========================================
Files 469 469
Lines 58298 58300 +2
==========================================
- Hits 48379 48375 -4
- Misses 9919 9925 +6
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/io/snowflake.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc25vd2ZsYWtlLnB5) | | |
| [...s/python/apache\_beam/testing/datatype\_inference.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9kYXRhdHlwZV9pbmZlcmVuY2UucHk=) | | |
| [...s/python/apache\_beam/testing/synthetic\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9zeW50aGV0aWNfcGlwZWxpbmUucHk=) | | |
| [...\_beam/io/gcp/datastore/v1new/adaptive\_throttler.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2RhdGFzdG9yZS92MW5ldy9hZGFwdGl2ZV90aHJvdHRsZXIucHk=) | | |
| [...\_beam/testing/benchmarks/nexmark/queries/query5.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTUucHk=) | | |
| [sdks/python/apache\_beam/runners/worker/logger.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvbG9nZ2VyLnB5) | | |
| [...python/apache\_beam/examples/wordcount\_debugging.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvd29yZGNvdW50X2RlYnVnZ2luZy5weQ==) | | |
| [...cp/internal/clients/storage/storage\_v1\_messages.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2ludGVybmFsL2NsaWVudHMvc3RvcmFnZS9zdG9yYWdlX3YxX21lc3NhZ2VzLnB5) | | |
| [.../python/apache\_beam/io/gcp/bigquery\_io\_metadata.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X2lvX21ldGFkYXRhLnB5) | | |
| [...on/apache\_beam/io/gcp/bigquery\_io\_read\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X2lvX3JlYWRfcGlwZWxpbmUucHk=) | | |
| ... and [928 more](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=footer). Last update [aaad864...5f8a15f](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784541216
# [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=h1) Report
> Merging [#14046](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=desc) (353e5db) into [master](https://codecov.io/gh/apache/beam/commit/aaad864c9acb22e35050f974a7ac74fb7638f085?el=desc) (aaad864) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #14046 +/- ##
==========================================
- Coverage 82.98% 82.97% -0.01%
==========================================
Files 469 469
Lines 58298 58300 +2
==========================================
- Hits 48379 48376 -3
- Misses 9919 9924 +5
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...hon/apache\_beam/runners/direct/direct\_userstate.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvZGlyZWN0X3VzZXJzdGF0ZS5weQ==) | | |
| [sdks/python/apache\_beam/typehints/decorators.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL2RlY29yYXRvcnMucHk=) | | |
| [...dks/python/apache\_beam/portability/api/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL19faW5pdF9fLnB5) | | |
| [...che\_beam/portability/api/beam\_expansion\_api\_pb2.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fZXhwYW5zaW9uX2FwaV9wYjIucHk=) | | |
| [...ache\_beam/portability/api/beam\_job\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fam9iX2FwaV9wYjJfZ3JwYy5weQ==) | | |
| [...s/snippets/transforms/aggregation/combinevalues.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9jb21iaW5ldmFsdWVzLnB5) | | |
| [...on/apache\_beam/runners/direct/watermark\_manager.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3Qvd2F0ZXJtYXJrX21hbmFnZXIucHk=) | | |
| [sdks/python/apache\_beam/utils/timestamp.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvdGltZXN0YW1wLnB5) | | |
| [sdks/python/apache\_beam/io/gcp/gcsio.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2djc2lvLnB5) | | |
| [sdks/python/apache\_beam/runners/runner.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9ydW5uZXIucHk=) | | |
| ... and [928 more](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=footer). Last update [aaad864...353e5db](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-785340175
thanks Sam!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784541216
# [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=h1) Report
> Merging [#14046](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=desc) (7a6b6bb) into [master](https://codecov.io/gh/apache/beam/commit/aaad864c9acb22e35050f974a7ac74fb7638f085?el=desc) (aaad864) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #14046 +/- ##
==========================================
- Coverage 82.98% 82.97% -0.01%
==========================================
Files 469 469
Lines 58298 58300 +2
==========================================
- Hits 48379 48376 -3
- Misses 9919 9924 +5
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/internal/pickler.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvcGlja2xlci5weQ==) | | |
| [sdks/python/apache\_beam/transforms/environments.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9lbnZpcm9ubWVudHMucHk=) | | |
| [...e\_beam/examples/complete/top\_wikipedia\_sessions.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvdG9wX3dpa2lwZWRpYV9zZXNzaW9ucy5weQ==) | | |
| [...beam/runners/interactive/display/pipeline\_graph.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3BpcGVsaW5lX2dyYXBoLnB5) | | |
| [sdks/python/apache\_beam/runners/worker/logger.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvbG9nZ2VyLnB5) | | |
| [sdks/python/apache\_beam/dataframe/convert.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2NvbnZlcnQucHk=) | | |
| [...apache\_beam/runners/dataflow/native\_io/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9uYXRpdmVfaW8vX19pbml0X18ucHk=) | | |
| [...python/apache\_beam/examples/wordcount\_xlang\_sql.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvd29yZGNvdW50X3hsYW5nX3NxbC5weQ==) | | |
| [...on/apache\_beam/runners/worker/statesampler\_slow.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc3RhdGVzYW1wbGVyX3Nsb3cucHk=) | | |
| [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | | |
| ... and [928 more](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=footer). Last update [aaad864...7a6b6bb](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14046: [BEAM-11791] Adding a microbenchmark for TestStream
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14046:
URL: https://github.com/apache/beam/pull/14046#issuecomment-784541216
# [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=h1) Report
> Merging [#14046](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=desc) (7a6b6bb) into [master](https://codecov.io/gh/apache/beam/commit/aaad864c9acb22e35050f974a7ac74fb7638f085?el=desc) (aaad864) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #14046 +/- ##
==========================================
- Coverage 82.98% 82.97% -0.01%
==========================================
Files 469 938 +469
Lines 58298 116600 +58302
==========================================
+ Hits 48379 96751 +48372
- Misses 9919 19849 +9930
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [.../apache\_beam/examples/cookbook/group\_with\_coder.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29va2Jvb2svZ3JvdXBfd2l0aF9jb2Rlci5weQ==) | | |
| [...che\_beam/examples/streaming\_wordcount\_debugging.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc3RyZWFtaW5nX3dvcmRjb3VudF9kZWJ1Z2dpbmcucHk=) | | |
| [sdks/python/apache\_beam/version.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdmVyc2lvbi5weQ==) | | |
| [sdks/python/apache\_beam/runners/runner.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9ydW5uZXIucHk=) | | |
| [sdks/python/apache\_beam/utils/sentinel.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc2VudGluZWwucHk=) | | |
| [sdks/python/apache\_beam/utils/plugin.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvcGx1Z2luLnB5) | | |
| [...che\_beam/runners/interactive/interactive\_runner.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9ydW5uZXIucHk=) | | |
| [...s/python/apache\_beam/examples/complete/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvX19pbml0X18ucHk=) | | |
| [sdks/python/apache\_beam/io/filesystems.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmlsZXN5c3RlbXMucHk=) | | |
| [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | | |
| ... and [1397 more](https://codecov.io/gh/apache/beam/pull/14046/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=footer). Last update [aaad864...7a6b6bb](https://codecov.io/gh/apache/beam/pull/14046?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org