You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/08/12 08:48:18 UTC
[GitHub] [beam] kamilwu opened a new pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
kamilwu opened a new pull request #12542:
URL: https://github.com/apache/beam/pull/12542
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2
--- | --- | --- | --- | --- | --- | ---
Go | [](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | ---
Java | [](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/)
Python | [](https://ci-beam.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/) | ---
XLang | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/) | ---
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #12542:
URL: https://github.com/apache/beam/pull/12542#issuecomment-672742021
Run Seed Job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on a change in pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu commented on a change in pull request #12542:
URL: https://github.com/apache/beam/pull/12542#discussion_r472013530
##########
File path: .test-infra/jenkins/job_LoadTests_coGBK_Python.groovy
##########
@@ -34,12 +34,12 @@ def loadTestConfigurations = { datasetName ->
pipelineOptions: [
project : 'apache-beam-testing',
region : 'us-central1',
- job_name : 'load-tests-python-dataflow-batch-cogbk-1-' + now,
+ job_name : "load-tests-python-dataflow-${mode}-cogbk-1-" + now,
Review comment:
Done :ok_hand:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on a change in pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu commented on a change in pull request #12542:
URL: https://github.com/apache/beam/pull/12542#discussion_r474018488
##########
File path: .test-infra/jenkins/job_LoadTests_coGBK_Python.groovy
##########
@@ -147,25 +147,30 @@ def loadTestConfigurations = { datasetName ->
autoscaling_algorithm: 'NONE'
]
],
- ].each { test -> test.pipelineOptions.putAll(additionalPipelineArgs) }
+ ]
+ .each { test -> test.pipelineOptions.putAll(additionalPipelineArgs) }
+ .each { test -> (mode) != 'streaming' ?: addStreamingOptions(test) }
}
-def batchLoadTestJob = { scope, triggeringContext ->
- scope.description('Runs Python CoGBK load tests on Dataflow runner in batch mode')
- commonJobProperties.setTopLevelMainJobProperties(scope, 'master', 240)
+def addStreamingOptions(test) {
+ // Use highmem workers to prevent out of memory issues.
+ test.pipelineOptions << [streaming: null,
+ worker_machine_type: 'n1-highmem-4'
Review comment:
> I wonder only what would happen if this worker won't be available when cron job starts?
Workers are still managed by Dataflow. That option (`--worker_machine_type`) is just a hint about what machines should Dataflow use
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kkucharc commented on a change in pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kkucharc commented on a change in pull request #12542:
URL: https://github.com/apache/beam/pull/12542#discussion_r469981965
##########
File path: .test-infra/jenkins/job_LoadTests_coGBK_Python.groovy
##########
@@ -25,7 +25,7 @@ import InfluxDBCredentialsHelper
def now = new Date().format("MMddHHmmss", TimeZone.getTimeZone('UTC'))
-def loadTestConfigurations = { datasetName ->
+def loadTestConfigurations = { mode, datasetName ->
Review comment:
I think we should agree on the common name of this param. In combine I put `jobType` (mainly because Java test has `jobType`), but there is a possibility to change it since combine isn't merged yet.
##########
File path: .test-infra/jenkins/job_LoadTests_coGBK_Python.groovy
##########
@@ -147,25 +147,30 @@ def loadTestConfigurations = { datasetName ->
autoscaling_algorithm: 'NONE'
]
],
- ].each { test -> test.pipelineOptions.putAll(additionalPipelineArgs) }
+ ]
+ .each { test -> test.pipelineOptions.putAll(additionalPipelineArgs) }
+ .each { test -> (mode) != 'streaming' ?: addStreamingOptions(test) }
}
-def batchLoadTestJob = { scope, triggeringContext ->
- scope.description('Runs Python CoGBK load tests on Dataflow runner in batch mode')
- commonJobProperties.setTopLevelMainJobProperties(scope, 'master', 240)
+def addStreamingOptions(test) {
+ // Use highmem workers to prevent out of memory issues.
+ test.pipelineOptions << [streaming: null,
+ worker_machine_type: 'n1-highmem-4'
Review comment:
Why this type of machine?
##########
File path: .test-infra/jenkins/job_LoadTests_coGBK_Python.groovy
##########
@@ -34,12 +34,12 @@ def loadTestConfigurations = { datasetName ->
pipelineOptions: [
project : 'apache-beam-testing',
region : 'us-central1',
- job_name : 'load-tests-python-dataflow-batch-cogbk-1-' + now,
+ job_name : "load-tests-python-dataflow-${mode}-cogbk-1-" + now,
Review comment:
Now you can also ass `now` as ${now} :D
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #12542:
URL: https://github.com/apache/beam/pull/12542#issuecomment-675353734
Run Load Tests Python CoGBK Dataflow Batch
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #12542:
URL: https://github.com/apache/beam/pull/12542#issuecomment-677716320
Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #12542:
URL: https://github.com/apache/beam/pull/12542#issuecomment-675353601
Run Load Tests Python CoGBK Dataflow Streaming
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu merged pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu merged pull request #12542:
URL: https://github.com/apache/beam/pull/12542
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on a change in pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu commented on a change in pull request #12542:
URL: https://github.com/apache/beam/pull/12542#discussion_r471557407
##########
File path: .test-infra/jenkins/job_LoadTests_coGBK_Python.groovy
##########
@@ -147,25 +147,30 @@ def loadTestConfigurations = { datasetName ->
autoscaling_algorithm: 'NONE'
]
],
- ].each { test -> test.pipelineOptions.putAll(additionalPipelineArgs) }
+ ]
+ .each { test -> test.pipelineOptions.putAll(additionalPipelineArgs) }
+ .each { test -> (mode) != 'streaming' ?: addStreamingOptions(test) }
}
-def batchLoadTestJob = { scope, triggeringContext ->
- scope.description('Runs Python CoGBK load tests on Dataflow runner in batch mode')
- commonJobProperties.setTopLevelMainJobProperties(scope, 'master', 240)
+def addStreamingOptions(test) {
+ // Use highmem workers to prevent out of memory issues.
+ test.pipelineOptions << [streaming: null,
+ worker_machine_type: 'n1-highmem-4'
Review comment:
My pipelines kept crushing because of OutOfMemory exceptions. So I followed the advice given in this article: https://cloud.google.com/community/tutorials/dataflow-debug-oom-conditions
Other solution was to, according to the article, use fewer number of threads per worker. But I couldn't have found a pipeline option responsible for that in Python SDK (it exists in Java SDK, though).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kkucharc commented on a change in pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kkucharc commented on a change in pull request #12542:
URL: https://github.com/apache/beam/pull/12542#discussion_r473985462
##########
File path: .test-infra/jenkins/job_LoadTests_coGBK_Python.groovy
##########
@@ -147,25 +147,30 @@ def loadTestConfigurations = { datasetName ->
autoscaling_algorithm: 'NONE'
]
],
- ].each { test -> test.pipelineOptions.putAll(additionalPipelineArgs) }
+ ]
+ .each { test -> test.pipelineOptions.putAll(additionalPipelineArgs) }
+ .each { test -> (mode) != 'streaming' ?: addStreamingOptions(test) }
}
-def batchLoadTestJob = { scope, triggeringContext ->
- scope.description('Runs Python CoGBK load tests on Dataflow runner in batch mode')
- commonJobProperties.setTopLevelMainJobProperties(scope, 'master', 240)
+def addStreamingOptions(test) {
+ // Use highmem workers to prevent out of memory issues.
+ test.pipelineOptions << [streaming: null,
+ worker_machine_type: 'n1-highmem-4'
Review comment:
I wonder only what would happen if this worker won't be available when cron job starts? Or it will be permanently shut down.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #12542:
URL: https://github.com/apache/beam/pull/12542#issuecomment-672747732
Run Spotless PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #12542:
URL: https://github.com/apache/beam/pull/12542#issuecomment-672872498
R: @kkucharc
cc: @tysonjh
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on a change in pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu commented on a change in pull request #12542:
URL: https://github.com/apache/beam/pull/12542#discussion_r474015631
##########
File path: .test-infra/jenkins/job_LoadTests_coGBK_Python.groovy
##########
@@ -147,25 +147,30 @@ def loadTestConfigurations = { datasetName ->
autoscaling_algorithm: 'NONE'
]
],
- ].each { test -> test.pipelineOptions.putAll(additionalPipelineArgs) }
+ ]
+ .each { test -> test.pipelineOptions.putAll(additionalPipelineArgs) }
+ .each { test -> (mode) != 'streaming' ?: addStreamingOptions(test) }
}
-def batchLoadTestJob = { scope, triggeringContext ->
- scope.description('Runs Python CoGBK load tests on Dataflow runner in batch mode')
- commonJobProperties.setTopLevelMainJobProperties(scope, 'master', 240)
+def addStreamingOptions(test) {
+ // Use highmem workers to prevent out of memory issues.
+ test.pipelineOptions << [streaming: null,
+ worker_machine_type: 'n1-highmem-4'
Review comment:
> How is it called in Java SDK?
It's `--numberOfWorkerHarnessThreads`. Some time ago, its equivalent in Python SDK was `--experimental worker_threads=[n]`, but the option was removed. I'll open a discussion on dev@ to find out if it is something the community should take care of.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kkucharc commented on a change in pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kkucharc commented on a change in pull request #12542:
URL: https://github.com/apache/beam/pull/12542#discussion_r474041812
##########
File path: .test-infra/jenkins/job_LoadTests_coGBK_Python.groovy
##########
@@ -147,25 +147,30 @@ def loadTestConfigurations = { datasetName ->
autoscaling_algorithm: 'NONE'
]
],
- ].each { test -> test.pipelineOptions.putAll(additionalPipelineArgs) }
+ ]
+ .each { test -> test.pipelineOptions.putAll(additionalPipelineArgs) }
+ .each { test -> (mode) != 'streaming' ?: addStreamingOptions(test) }
}
-def batchLoadTestJob = { scope, triggeringContext ->
- scope.description('Runs Python CoGBK load tests on Dataflow runner in batch mode')
- commonJobProperties.setTopLevelMainJobProperties(scope, 'master', 240)
+def addStreamingOptions(test) {
+ // Use highmem workers to prevent out of memory issues.
+ test.pipelineOptions << [streaming: null,
+ worker_machine_type: 'n1-highmem-4'
Review comment:
Thanks a lot Kamil :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kkucharc commented on a change in pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kkucharc commented on a change in pull request #12542:
URL: https://github.com/apache/beam/pull/12542#discussion_r473983538
##########
File path: .test-infra/jenkins/job_LoadTests_coGBK_Python.groovy
##########
@@ -147,25 +147,30 @@ def loadTestConfigurations = { datasetName ->
autoscaling_algorithm: 'NONE'
]
],
- ].each { test -> test.pipelineOptions.putAll(additionalPipelineArgs) }
+ ]
+ .each { test -> test.pipelineOptions.putAll(additionalPipelineArgs) }
+ .each { test -> (mode) != 'streaming' ?: addStreamingOptions(test) }
}
-def batchLoadTestJob = { scope, triggeringContext ->
- scope.description('Runs Python CoGBK load tests on Dataflow runner in batch mode')
- commonJobProperties.setTopLevelMainJobProperties(scope, 'master', 240)
+def addStreamingOptions(test) {
+ // Use highmem workers to prevent out of memory issues.
+ test.pipelineOptions << [streaming: null,
+ worker_machine_type: 'n1-highmem-4'
Review comment:
I see. Maybe it's worth to create an issue in jira with this bug? I am not sure it should be infra issue or maybe python if we need this option. How is it called in Java SDK?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kkucharc commented on pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kkucharc commented on pull request #12542:
URL: https://github.com/apache/beam/pull/12542#issuecomment-677712850
This LGTM 💯 Thank you a lot @kamilwu for your contribution
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #12542:
URL: https://github.com/apache/beam/pull/12542#issuecomment-672790408
Run Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #12542:
URL: https://github.com/apache/beam/pull/12542#issuecomment-672824673
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #12542:
URL: https://github.com/apache/beam/pull/12542#issuecomment-672752217
Run Load Tests Python CoGBK Dataflow Batch
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #12542:
URL: https://github.com/apache/beam/pull/12542#issuecomment-675343972
Run Seed Job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #12542:
URL: https://github.com/apache/beam/pull/12542#issuecomment-672752037
Run Load Tests Python CoGBK Dataflow Streaming
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #12542:
URL: https://github.com/apache/beam/pull/12542#issuecomment-672820207
Run Seed Job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on a change in pull request #12542: [BEAM-10674] Add Python CoGBK load test for streaming on Dataflow
Posted by GitBox <gi...@apache.org>.
kamilwu commented on a change in pull request #12542:
URL: https://github.com/apache/beam/pull/12542#discussion_r471559738
##########
File path: .test-infra/jenkins/job_LoadTests_coGBK_Python.groovy
##########
@@ -25,7 +25,7 @@ import InfluxDBCredentialsHelper
def now = new Date().format("MMddHHmmss", TimeZone.getTimeZone('UTC'))
-def loadTestConfigurations = { datasetName ->
+def loadTestConfigurations = { mode, datasetName ->
Review comment:
After an offline discussion, we decided to stay with `mode` in this context.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org