You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "bambadiouf1 (via GitHub)" <gi...@apache.org> on 2026/01/21 20:56:35 UTC
[PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
bambadiouf1 opened a new pull request, #37377:
URL: https://github.com/apache/beam/pull/37377
**Please** add a meaningful description for your change here
This pull request introduces two new pipeline options for the Google Cloud Dataflow runner across the Python, Java, and Go SDKs ([ source](https://github.com/apache/beam/compare/master...bambadiouf1:beam:iops_throughput_pipeline_options?expand=1&content_ref=add+diskprovisionediops+and+diskprovisionedthroughputmibps+pipeline+o+ptions+for+python+java+and+go+sdks#diff-68a845ae48fadf50528918b93dea4a227e0572038fb54ec68c79e68f11b51ac5)). These options allow users to specify provisioned performance for worker VM root disks:
disk_provisioned_iops: Sets the provisioned IOPS for the root disk. If unspecified, the service chooses a default ([ source](https://github.com/apache/beam/compare/master...bambadiouf1:beam:iops_throughput_pipeline_options?expand=1&content_ref=iops+provisioned+for+the+root+disk+for+vms+if+zero+or+unspecified+the+service+will+attempt+to+choose+a+reasonable+default#diff-68a845ae48fadf50528918b93dea4a227e0572038fb54ec68c79e68f11b51ac5)).
disk_provisioned_throughput_mibps: Sets the provisioned throughput in MiB/s for the root disk ([ source](https://github.com/apache/beam/compare/master...bambadiouf1:beam:iops_throughput_pipeline_options?expand=1&content_ref=for+the+root+disk+for+vms+if+zero+or+unspecified+the+service+will+attempt+to+choose+a+reasonable+default#diff-68a845ae48fadf50528918b93dea4a227e0572038fb54ec68c79e68f11b51ac5)).
Changes by SDK:
Java SDK: Added getters and setters to DataflowPipelineWorkerPoolOptions and updated DataflowPipelineTranslator to pass these values to the worker pool configuration ([ source](https://github.com/apache/beam/compare/master...bambadiouf1:beam:iops_throughput_pipeline_options?expand=1&content_ref=workerpool+setdiskprovisionediops+options+getdiskprovisionediops+if+options+getdiskprovisionedthroughputmibps#diff-68a845ae48fadf50528918b93dea4a227e0572038fb54ec68c79e68f11b51ac5)).
Python SDK: Added --disk_provisioned_iops and --disk_provisioned_throughput_mibps to WorkerOptions and updated the internal apiclient to include these fields in the Dataflow environment ([ source](https://github.com/apache/beam/compare/master...bambadiouf1:beam:iops_throughput_pipeline_options?expand=1&content_ref=worker_options+disk_provisioned_iops+pool+diskprovisionediops+self+worker_options+disk_provisioned_iops#diff-68a845ae48fadf50528918b93dea4a227e0572038fb54ec68c79e68f11b51ac5)).
Go SDK: Added flags for both options and updated the dataflowlib job translation logic ([ source](https://github.com/apache/beam/compare/master...bambadiouf1:beam:iops_throughput_pipeline_options?expand=1&content_ref=diskprovisionediops+flag+int64+disk_provisioned_iops+0+provisioned+iops+for+the+root+disk+for+vms+optional#diff-68a845ae48fadf50528918b93dea4a227e0572038fb54ec68c79e68f11b51ac5)).
Tests have been added to verify that these options are correctly parsed and translated for each SDK ([ source](https://github.com/apache/beam/compare/master...bambadiouf1:beam:iops_throughput_pipeline_options?expand=1&content_ref=pelineoptions+class+options+setdiskprovisionediops+long+valueof+7000+options+setdiskprovisionedthroughputmibps+long+valueof+250+options+setproject+test+project#diff-68a845ae48fadf50528918b93dea4a227e0572038fb54ec68c79e68f11b51ac5)).
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37377:
URL: https://github.com/apache/beam/pull/37377#issuecomment-3781297438
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops/ThroughputMibps options and update Go client libraries [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #37377:
URL: https://github.com/apache/beam/pull/37377#issuecomment-4344967465
> Hey, this PR actually breaks the Dataflow protos in python by changing field numbers for existing fields. Additionally, this is changing the internally generated python client that we're in the process of deprecating. I'd suggest reverting this PR and removing the Python-specific code
This is my fault, I should've caught the conflict. The relevant PR making this change is https://github.com/apache/beam/pull/37639, which would basically need to undo all the Python pieces here.
I agree, lets:
1) Revert
2) Put up another PR with just the Java and Go side changes. We can merge that immediately.
3) Address the Python pieces separately. @jrmccluskey could you help Bamba with that after you merge your PR? (either by just doing it or providing code pointers on what changes need to happen)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "bambadiouf1 (via GitHub)" <gi...@apache.org>.
bambadiouf1 commented on PR #37377:
URL: https://github.com/apache/beam/pull/37377#issuecomment-4241076192
assign set of reviewers
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #37377:
URL: https://github.com/apache/beam/pull/37377#discussion_r3150003494
##########
sdks/python/apache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_messages.py:
##########
@@ -8047,11 +8062,3 @@ class WriteInstruction(_messages.Message):
StandardQueryParameters.FXgafvValueValuesEnum, '_1', '1')
encoding.AddCustomJsonEnumMapping(
StandardQueryParameters.FXgafvValueValuesEnum, '_2', '2')
-encoding.AddCustomJsonFieldMapping(
- DataflowProjectsLocationsTemplatesLaunchRequest, 'dynamicTemplate_gcsPath', 'dynamicTemplate.gcsPath')
-encoding.AddCustomJsonFieldMapping(
- DataflowProjectsLocationsTemplatesLaunchRequest, 'dynamicTemplate_stagingLocation', 'dynamicTemplate.stagingLocation')
-encoding.AddCustomJsonFieldMapping(
- DataflowProjectsTemplatesLaunchRequest, 'dynamicTemplate_gcsPath', 'dynamicTemplate.gcsPath')
-encoding.AddCustomJsonFieldMapping(
- DataflowProjectsTemplatesLaunchRequest, 'dynamicTemplate_stagingLocation', 'dynamicTemplate.stagingLocation')
Review Comment:
I think this is deleted because of a merge issue (this was added after the commit you're generating from) - could you try pulling in the latest contents of master (`git pull origin master`) and then regenerating this client?
##########
sdks/go.mod:
##########
@@ -25,12 +25,12 @@ go 1.26.0
toolchain go1.26.2
require (
- cloud.google.com/go/bigquery v1.72.0
- cloud.google.com/go/bigtable v1.41.0
- cloud.google.com/go/datastore v1.21.0
+ cloud.google.com/go/bigquery v1.74.0
+ cloud.google.com/go/bigtable v1.42.0
+ cloud.google.com/go/datastore v1.22.0
Review Comment:
Oh, and could you also mention the pipeline options in https://github.com/apache/beam/blob/master/CHANGES.md ? (you don't need to call out the client libraries there)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #37377:
URL: https://github.com/apache/beam/pull/37377#issuecomment-4321116569
## [Codecov](https://app.codecov.io/gh/apache/beam/pull/37377?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 71.90%. Comparing base ([`a615657`](https://app.codecov.io/gh/apache/beam/commit/a615657d5a41a0baa5b653abe072fc4c1c9dcd6c?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`785d55c`](https://app.codecov.io/gh/apache/beam/commit/785d55c8fb303ffc6142d64e3086554af8f44b36?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #37377 +/- ##
=============================================
+ Coverage 58.52% 71.90% +13.37%
+ Complexity 15434 3425 -12009
=============================================
Files 2851 329 -2522
Lines 280086 29836 -250250
Branches 12337 3593 -8744
=============================================
- Hits 163917 21453 -142464
+ Misses 109756 6739 -103017
+ Partials 6413 1644 -4769
```
| [Flag](https://app.codecov.io/gh/apache/beam/pull/37377/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [go](https://app.codecov.io/gh/apache/beam/pull/37377/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java](https://app.codecov.io/gh/apache/beam/pull/37377/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `71.90% <ø> (+7.29%)` | :arrow_up: |
| [python](https://app.codecov.io/gh/apache/beam/pull/37377/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/beam/pull/37377?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #37377:
URL: https://github.com/apache/beam/pull/37377#issuecomment-4330165411
/gemini review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops/ThroughputMibps options and update Go client libraries [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #37377:
URL: https://github.com/apache/beam/pull/37377#issuecomment-4335915617
/gemini review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on code in PR #37377:
URL: https://github.com/apache/beam/pull/37377#discussion_r3133415730
##########
runners/google-cloud-dataflow-java/src/test/java/org/apache/beam/runners/dataflow/options/DataflowPipelineOptionsTest.java:
##########
@@ -322,4 +322,13 @@ public void destroy() {
TimeoutException.class, () -> DefaultGcpRegionFactory.getRegionFromGcloudCli(1L));
}
}
+
+ @Test
+ public void testDiskProvisionedIopsAndThroughput() {
+ DataflowPipelineOptions options = PipelineOptionsFactory.as(DataflowPipelineOptions.class);
+ options.setDiskProvisionedIops(1000);
+ options.setDiskProvisionedThroughputMibps(100);
+ assertEquals(Integer.valueOf(1000), options.getDiskProvisionedIops());
+ assertEquals(Integer.valueOf(100), options.getDiskProvisionedThroughputMibps());
Review Comment:
These assertions will fail because `options.getDiskProvisionedIops()` returns a `Long`, while `Integer.valueOf(1000)` creates an `Integer`. In Java, an `Integer` object is never equal to a `Long` object via `.equals()`. You should use `Long.valueOf()` or a long literal (e.g., `1000L`) to match the return type of the getter.
```suggestion
assertEquals(Long.valueOf(1000), options.getDiskProvisionedIops());
assertEquals(Long.valueOf(100), options.getDiskProvisionedThroughputMibps());
```
##########
runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineTranslator.java:
##########
@@ -489,6 +489,13 @@ public Job translate(List<DataflowPackage> packages) {
if (options.getDiskSizeGb() > 0) {
workerPool.setDiskSizeGb(options.getDiskSizeGb());
}
+ if (options.getDiskProvisionedIops() != null && options.getDiskProvisionedIops() > 0) {
+ workerPool.setDiskProvisionedIops((long) options.getDiskProvisionedIops());
+ }
+ if (options.getDiskProvisionedThroughputMibps() != null && options.getDiskProvisionedThroughputMibps() > 0) {
+ workerPool.setDiskProvisionedThroughputMibps(
+ (long) options.getDiskProvisionedThroughputMibps());
+ }
Review Comment:
The cast to `(long)` is likely incorrect and may cause a compilation error. Based on the Dataflow API definition (and the Python SDK changes in this PR using `INT32`), these fields are typically 32-bit integers. Since `options.getDiskProvisionedIops()` returns a `Long`, you should convert it to an `Integer` (e.g., using `.intValue()`) before passing it to the `workerPool` setter, assuming the generated client library expects an `Integer` for these fields. Alternatively, consider changing the option type to `Integer` in `DataflowPipelineWorkerPoolOptions` for consistency with other fields like `diskSizeGb`.
```suggestion
if (options.getDiskProvisionedIops() != null && options.getDiskProvisionedIops() > 0) {
workerPool.setDiskProvisionedIops(options.getDiskProvisionedIops().intValue());
}
if (options.getDiskProvisionedThroughputMibps() != null && options.getDiskProvisionedThroughputMibps() > 0) {
workerPool.setDiskProvisionedThroughputMibps(
options.getDiskProvisionedThroughputMibps().intValue());
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops/ThroughputMibps options and update Go client libraries [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on PR #37377:
URL: https://github.com/apache/beam/pull/37377#issuecomment-4392831925
> Hey, this PR actually breaks the Dataflow protos in python by changing field numbers for existing fields
I think apitools generates the fields in the proto alphabetically, and the order likely is not material for the underlying JSON api; we made similar updates in https://github.com/apache/beam/pull/37581/changes and earlier changes.
Did something get broken by this PR when it was merge or we reverted out of caution?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37377:
URL: https://github.com/apache/beam/pull/37377#issuecomment-4241080868
Assigning reviewers:
R: @damccorm for label python.
R: @jrmccluskey for label go.
R: @Abacn for label build.
Note: If you would like to opt out of this review, comment `assign to next reviewer`.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on code in PR #37377:
URL: https://github.com/apache/beam/pull/37377#discussion_r3079455108
##########
sdks/go/pkg/beam/runners/dataflow/dataflow.go:
##########
@@ -61,9 +61,11 @@ var (
numWorkers = flag.Int64("num_workers", 0, "Number of workers (optional).")
workerHarnessThreads = flag.Int64("number_of_worker_harness_threads", 0, "The number of threads per each worker harness process (optional).")
maxNumWorkers = flag.Int64("max_num_workers", 0, "Maximum number of workers during scaling (optional).")
- diskSizeGb = flag.Int64("disk_size_gb", 0, "Size of root disk for VMs, in GB (optional).")
- diskType = flag.String("disk_type", "", "Type of root disk for VMs (optional).")
- autoscalingAlgorithm = flag.String("autoscaling_algorithm", "", "Autoscaling mode to use (optional).")
+ diskSizeGb = flag.Int64("disk_size_gb", 0, "Size of root disk for VMs, in GB (optional).")
+ diskType = flag.String("disk_type", "", "Type of root disk for VMs (optional).")
+ diskProvisionedIops = flag.Int64("disk_provisioned_iops", 0, "Provisioned IOPS for the worker disk (optional).")
+ diskProvisionedThroughputMibps = flag.Int64("disk_provisioned_throughput_mibps", 0, "Provisioned throughput in MiB/s for the worker disk (optional).")
Review Comment:
The Go implementation appears to be incomplete. While flags and `JobOptions` fields have been added, there is no corresponding logic to translate these values into the `WorkerPool` configuration within the Dataflow API request (typically handled in `environment.go`). Without this translation, these flags will have no effect on the provisioned workers.
##########
runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/options/DataflowPipelineWorkerPoolOptions.java:
##########
@@ -193,6 +193,20 @@ public String getAlgorithm() {
void setWorkerDiskType(String value);
+ /** Provisioned IOPS for the worker disk. */
+ @Description("Provisioned IOPS for the worker disk.")
+ @Nullable
+ Integer getDiskProvisionedIops();
+
+ void setDiskProvisionedIops(Integer value);
Review Comment:
Consider using `Long` instead of `Integer` for these performance-related fields. While `Integer` is likely sufficient for current IOPS limits, using `Long` is more consistent with the underlying Dataflow API (which the translator casts to `Long`) and the Go implementation in this PR. It also future-proofs the options against very high performance disk specifications.
```suggestion
Long getDiskProvisionedIops();
void setDiskProvisionedIops(Long value);
```
##########
runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineTranslator.java:
##########
@@ -489,6 +489,13 @@ public Job translate(List<DataflowPackage> packages) {
if (options.getDiskSizeGb() > 0) {
workerPool.setDiskSizeGb(options.getDiskSizeGb());
}
+ if (options.getDiskProvisionedIops() != null) {
+ workerPool.setDiskProvisionedIops(Long.valueOf(options.getDiskProvisionedIops()));
+ }
+ if (options.getDiskProvisionedThroughputMibps() != null) {
+ workerPool.setDiskProvisionedThroughputMibps(
+ Long.valueOf(options.getDiskProvisionedThroughputMibps()));
+ }
Review Comment:
For consistency with how `diskSizeGb` is handled (lines 489-491), it is recommended to also check that these values are greater than zero. This provides basic client-side validation and ensures that a value of 0 (if provided) doesn't inadvertently override service defaults in an unexpected way. Additionally, the conversion to `Long` can be simplified.
```suggestion
if (options.getDiskProvisionedIops() != null && options.getDiskProvisionedIops() > 0) {
workerPool.setDiskProvisionedIops((long) options.getDiskProvisionedIops());
}
if (options.getDiskProvisionedThroughputMibps() != null && options.getDiskProvisionedThroughputMibps() > 0) {
workerPool.setDiskProvisionedThroughputMibps(
(long) options.getDiskProvisionedThroughputMibps());
}
```
##########
runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/options/DataflowPipelineWorkerPoolOptions.java:
##########
@@ -193,6 +193,20 @@ public String getAlgorithm() {
void setWorkerDiskType(String value);
+ /** Provisioned IOPS for the worker disk. */
+ @Description("Provisioned IOPS for the worker disk.")
+ @Nullable
+ Integer getDiskProvisionedIops();
+
+ void setDiskProvisionedIops(Integer value);
+
+ /** Provisioned throughput in MiB/s for the worker disk. */
+ @Description("Provisioned throughput in MiB/s for the worker disk.")
+ @Nullable
+ Integer getDiskProvisionedThroughputMibps();
+
+ void setDiskProvisionedThroughputMibps(Integer value);
Review Comment:
Consider using `Long` for throughput as well, for consistency and future-proofing.
```suggestion
Long getDiskProvisionedThroughputMibps();
void setDiskProvisionedThroughputMibps(Long value);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "bambadiouf1 (via GitHub)" <gi...@apache.org>.
bambadiouf1 commented on code in PR #37377:
URL: https://github.com/apache/beam/pull/37377#discussion_r3127636344
##########
sdks/go/pkg/beam/runners/dataflow/dataflowlib/job.go:
##########
@@ -57,8 +57,10 @@ type JobOptions struct {
NoUsePublicIPs bool
NumWorkers int64
DiskSizeGb int64
- DiskType string
- MachineType string
+ DiskType string
+ DiskProvisionedIops int64
+ DiskProvisionedThroughputMibps int64
+ MachineType string
Review Comment:
Thanks, Danny! I ran `got fmt` and the all format fixes have been applied :).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops/ThroughputMibps options and update Go client libraries [beam]
Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on code in PR #37377:
URL: https://github.com/apache/beam/pull/37377#discussion_r3154741469
##########
CHANGES.md:
##########
@@ -68,6 +68,7 @@
## New Features / Improvements
+* Added support for setting disk provisioned IOPS and throughput in Dataflow runner via `--diskProvisionedIops` and `--diskProvisionedThroughputMibps` pipeline options (Java/Python/Go) ([#37377](https://github.com/apache/beam/issues/37377)).
Review Comment:
The flag names listed here (`--diskProvisionedIops` and `--diskProvisionedThroughputMibps`) are only accurate for the Java SDK. In Python and Go, these pipeline options use snake_case: `--disk_provisioned_iops` and `--disk_provisioned_throughput_mibps`. Please update the description to be accurate for all supported languages.
```suggestion
* Added support for setting disk provisioned IOPS and throughput in Dataflow runner (Java/Python/Go) ([#37377](https://github.com/apache/beam/issues/37377)).
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #37377:
URL: https://github.com/apache/beam/pull/37377#issuecomment-4307428941
/gemini review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37377:
URL: https://github.com/apache/beam/pull/37377#issuecomment-4296259429
Reminder, please take a look at this pr: @damccorm @jrmccluskey @Abacn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops/ThroughputMibps options and update Go client libraries [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #37377:
URL: https://github.com/apache/beam/pull/37377#discussion_r3154858198
##########
CHANGES.md:
##########
@@ -68,6 +68,7 @@
## New Features / Improvements
+* Added support for setting disk provisioned IOPS and throughput in Dataflow runner via `--diskProvisionedIops` and `--diskProvisionedThroughputMibps` pipeline options (Java/Python/Go) ([#37377](https://github.com/apache/beam/issues/37377)).
Review Comment:
This is fine, it conveys the idea
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops/ThroughputMibps options and update Go client libraries [beam]
Posted by "bambadiouf1 (via GitHub)" <gi...@apache.org>.
bambadiouf1 commented on code in PR #37377:
URL: https://github.com/apache/beam/pull/37377#discussion_r3150428175
##########
sdks/go.mod:
##########
@@ -25,12 +25,12 @@ go 1.26.0
toolchain go1.26.2
require (
- cloud.google.com/go/bigquery v1.72.0
- cloud.google.com/go/bigtable v1.41.0
- cloud.google.com/go/datastore v1.21.0
+ cloud.google.com/go/bigquery v1.74.0
+ cloud.google.com/go/bigtable v1.42.0
+ cloud.google.com/go/datastore v1.22.0
Review Comment:
Thanks, Danny! I updated the PR's title and description accordingly.
I also updated the upcoming 2.74.0 release section in https://github.com/apache/beam/blob/master/CHANGES.md to include the pipeline options.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops/ThroughputMibps options and update Go client libraries [beam]
Posted by "bambadiouf1 (via GitHub)" <gi...@apache.org>.
bambadiouf1 commented on code in PR #37377:
URL: https://github.com/apache/beam/pull/37377#discussion_r3150343163
##########
sdks/python/apache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_messages.py:
##########
@@ -8047,11 +8062,3 @@ class WriteInstruction(_messages.Message):
StandardQueryParameters.FXgafvValueValuesEnum, '_1', '1')
encoding.AddCustomJsonEnumMapping(
StandardQueryParameters.FXgafvValueValuesEnum, '_2', '2')
-encoding.AddCustomJsonFieldMapping(
- DataflowProjectsLocationsTemplatesLaunchRequest, 'dynamicTemplate_gcsPath', 'dynamicTemplate.gcsPath')
-encoding.AddCustomJsonFieldMapping(
- DataflowProjectsLocationsTemplatesLaunchRequest, 'dynamicTemplate_stagingLocation', 'dynamicTemplate.stagingLocation')
-encoding.AddCustomJsonFieldMapping(
- DataflowProjectsTemplatesLaunchRequest, 'dynamicTemplate_gcsPath', 'dynamicTemplate.gcsPath')
-encoding.AddCustomJsonFieldMapping(
- DataflowProjectsTemplatesLaunchRequest, 'dynamicTemplate_stagingLocation', 'dynamicTemplate.stagingLocation')
Review Comment:
Thanks a lot, Danny! I did a pull origin but the regenerating the gen_client causes these lines to be deleted for some reasons, so I reverted the changes!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops/ThroughputMibps options and update Go client libraries [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm merged PR #37377:
URL: https://github.com/apache/beam/pull/37377
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on PR #37377:
URL: https://github.com/apache/beam/pull/37377#issuecomment-3781128628
## Summary of Changes
Hello @bambadiouf1, I'm Gemini Code Assist[^1]! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request enhances Google Cloud Dataflow's capabilities by allowing users to specify detailed performance characteristics for worker VM root disks. By introducing options for provisioned IOPS and throughput, it provides greater control over the underlying infrastructure, which can be crucial for optimizing the performance and cost-efficiency of data processing pipelines. This change is integrated across all supported SDKs, ensuring a consistent experience for developers.
### Highlights
* **New Pipeline Options**: Introduced `DiskProvisionedIops` and `DiskProvisionedThroughputMibps` as new pipeline options for Google Cloud Dataflow.
* **Cross-SDK Implementation**: These new options are implemented across the Python, Java, and Go SDKs, allowing users to specify provisioned performance for worker VM root disks regardless of the SDK used.
* **Worker Disk Performance Control**: Users can now explicitly set the provisioned IOPS (Input/Output Operations Per Second) and throughput (in MiB/s) for Dataflow worker root disks, offering finer control over job performance.
* **Comprehensive Testing**: New tests have been added for each SDK to ensure that these options are correctly parsed, translated, and applied to the Dataflow worker pool configuration.
🧠 **New Feature in Public Preview:** You can now enable **Memory** to help **Gemini Code Assist** learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. **Click [here](https://codeassist.google/code-review/login) to enable Memory in your admin console.**
<details>
<summary><b>Using Gemini Code Assist</b></summary>
<br>
The full guide for Gemini Code Assist can be found on our [documentation page](https://developers.google.com/gemini-code-assist/docs/review-github-code), here are some quick tips.
<b>Invoking Gemini</b>
You can request assistance from Gemini at any point by creating a comment using either `/gemini <command>` or `@gemini-code-assist <command>`. Below is a summary of the supported commands on the current page.
Feature | Command | Description
--- | --- | ---
Code Review | `/gemini review` | Performs a code review for the current pull request in its current state.
Pull Request Summary | `/gemini summary` | Provides a summary of the current pull request in its current state.
Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help | `/gemini help` | Displays a list of available commands.
<b>Customization</b>
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a `.gemini/` folder in the base of the repository. Detailed instructions can be found [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github).
<b>Limitations & Feedback</b>
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with :thumbsup: and :thumbsdown: on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up [here](https://google.qualtrics.com/jfe/form/SV_2cyuGuTWsEw84yG).
<b>You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the [Gemini Code Assist IDE Extension](https://cloud.google.com/products/gemini/code-assist).</b>
</details>
[^1]: Review the [Privacy Notices](https://policies.google.com/privacy), [Generative AI Prohibited Use Policy](https://policies.google.com/terms/generative-ai/use-policy), [Terms of Service](https://policies.google.com/terms), and learn how to configure Gemini Code Assist in GitHub [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github). Gemini can make mistakes, so double check it and [use code with caution](https://support.google.com/legal/answer/13505487).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #37377:
URL: https://github.com/apache/beam/pull/37377#issuecomment-4300581789
waiting on author
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #37377:
URL: https://github.com/apache/beam/pull/37377#issuecomment-4243886243
/gemini review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #37377:
URL: https://github.com/apache/beam/pull/37377#discussion_r3133414460
##########
sdks/go.mod:
##########
@@ -25,12 +25,12 @@ go 1.26.0
toolchain go1.26.2
require (
- cloud.google.com/go/bigquery v1.72.0
- cloud.google.com/go/bigtable v1.41.0
- cloud.google.com/go/datastore v1.21.0
+ cloud.google.com/go/bigquery v1.74.0
+ cloud.google.com/go/bigtable v1.42.0
+ cloud.google.com/go/datastore v1.22.0
Review Comment:
Do we need all these dependency changes? I think they're maybe responsible for precommit failures
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #37377:
URL: https://github.com/apache/beam/pull/37377#discussion_r3079462008
##########
sdks/go/pkg/beam/runners/dataflow/dataflowlib/job.go:
##########
@@ -57,8 +57,10 @@ type JobOptions struct {
NoUsePublicIPs bool
NumWorkers int64
DiskSizeGb int64
- DiskType string
- MachineType string
+ DiskType string
+ DiskProvisionedIops int64
+ DiskProvisionedThroughputMibps int64
+ MachineType string
Review Comment:
These don't look correctly formatted - could you please run `go fmt` to fix?
##########
sdks/go/pkg/beam/runners/dataflow/dataflow.go:
##########
@@ -61,9 +61,11 @@ var (
numWorkers = flag.Int64("num_workers", 0, "Number of workers (optional).")
workerHarnessThreads = flag.Int64("number_of_worker_harness_threads", 0, "The number of threads per each worker harness process (optional).")
maxNumWorkers = flag.Int64("max_num_workers", 0, "Maximum number of workers during scaling (optional).")
- diskSizeGb = flag.Int64("disk_size_gb", 0, "Size of root disk for VMs, in GB (optional).")
- diskType = flag.String("disk_type", "", "Type of root disk for VMs (optional).")
- autoscalingAlgorithm = flag.String("autoscaling_algorithm", "", "Autoscaling mode to use (optional).")
+ diskSizeGb = flag.Int64("disk_size_gb", 0, "Size of root disk for VMs, in GB (optional).")
+ diskType = flag.String("disk_type", "", "Type of root disk for VMs (optional).")
+ diskProvisionedIops = flag.Int64("disk_provisioned_iops", 0, "Provisioned IOPS for the worker disk (optional).")
+ diskProvisionedThroughputMibps = flag.Int64("disk_provisioned_throughput_mibps", 0, "Provisioned throughput in MiB/s for the worker disk (optional).")
Review Comment:
I agree that I would've expected this to show up in https://github.com/apache/beam/blob/3f5ce80418be222c4243e2003041048041ae54e0/sdks/go/pkg/beam/runners/dataflow/dataflowlib/job.go#L188
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops/ThroughputMibps options and update Go client libraries [beam]
Posted by "bambadiouf1 (via GitHub)" <gi...@apache.org>.
bambadiouf1 commented on PR #37377:
URL: https://github.com/apache/beam/pull/37377#issuecomment-4339479769
> Thanks! Retrying some precommits to try to get a green signal, but I will eventually merge (unless they continue to fail and seem to represent a real problem)
Thanks a lot, Danny :)!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops/ThroughputMibps options and update Go client libraries [beam]
Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey commented on PR #37377:
URL: https://github.com/apache/beam/pull/37377#issuecomment-4344688579
Hey, this PR actually breaks the Dataflow protos in python by changing field numbers for existing fields. Additionally, this is changing the internally generated python client that we're in the process of deprecating. I'd suggest reverting this PR and removing the Python-specific code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "bambadiouf1 (via GitHub)" <gi...@apache.org>.
bambadiouf1 commented on code in PR #37377:
URL: https://github.com/apache/beam/pull/37377#discussion_r3127889142
##########
sdks/go/pkg/beam/runners/dataflow/dataflow.go:
##########
@@ -61,9 +61,11 @@ var (
numWorkers = flag.Int64("num_workers", 0, "Number of workers (optional).")
workerHarnessThreads = flag.Int64("number_of_worker_harness_threads", 0, "The number of threads per each worker harness process (optional).")
maxNumWorkers = flag.Int64("max_num_workers", 0, "Maximum number of workers during scaling (optional).")
- diskSizeGb = flag.Int64("disk_size_gb", 0, "Size of root disk for VMs, in GB (optional).")
- diskType = flag.String("disk_type", "", "Type of root disk for VMs (optional).")
- autoscalingAlgorithm = flag.String("autoscaling_algorithm", "", "Autoscaling mode to use (optional).")
+ diskSizeGb = flag.Int64("disk_size_gb", 0, "Size of root disk for VMs, in GB (optional).")
+ diskType = flag.String("disk_type", "", "Type of root disk for VMs (optional).")
+ diskProvisionedIops = flag.Int64("disk_provisioned_iops", 0, "Provisioned IOPS for the worker disk (optional).")
+ diskProvisionedThroughputMibps = flag.Int64("disk_provisioned_throughput_mibps", 0, "Provisioned throughput in MiB/s for the worker disk (optional).")
Review Comment:
Thanks for catching that, Danny! I've added the IOPS and throughput fields to the WorkerPool. I also updated the `google.golang.org/api` dependency to the latest version, using ` go get google.golang.org/api@latest` so these fields are available. It seems like the other required dependencies were updated for compatibility as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add DiskProvisionedIops and DiskProvisionedThroughputMibps pipeline options [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #37377:
URL: https://github.com/apache/beam/pull/37377#discussion_r3149966576
##########
sdks/go.mod:
##########
@@ -25,12 +25,12 @@ go 1.26.0
toolchain go1.26.2
require (
- cloud.google.com/go/bigquery v1.72.0
- cloud.google.com/go/bigtable v1.41.0
- cloud.google.com/go/datastore v1.21.0
+ cloud.google.com/go/bigquery v1.74.0
+ cloud.google.com/go/bigtable v1.42.0
+ cloud.google.com/go/datastore v1.22.0
Review Comment:
Ok - my guess is not everything got fully reverted since these shouldn't be needed for the change; with that said, the changes are fine/helpful and I see you fixed the fhir issue.
Would you mind updating the PR title to `Add DiskProvisionedIops/ThroughputMibps options and updated Go client libraries` and then updating the description to mention this change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org