You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by GitBox <gi...@apache.org> on 2020/02/28 03:04:54 UTC
[GitHub] [beam] angoenka opened a new pull request #10998: [BEAM-9287] Add
Postcommit tests for dataflow runner v2
angoenka opened a new pull request #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998
with hardcoded image for now
**Please** add a meaningful description for your change here
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] angoenka merged pull request #10998: [BEAM-9287] Add
Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
angoenka merged pull request #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] angoenka commented on issue #10998: [BEAM-9287] Add
Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
angoenka commented on issue #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#issuecomment-592696080
Run Python Dataflow V2 ValidatesRunner
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] ananvay commented on a change in pull request #10998:
[BEAM-9287] Add Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
ananvay commented on a change in pull request #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#discussion_r385488485
##########
File path: .test-infra/jenkins/job_PostCommit_Python_ValidatesRunner_Dataflow_V2.groovy
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+import PostcommitJobBuilder
+
+// This job runs the suite of Python ValidatesRunner tests against the
+// Dataflow runner V2.
+PostcommitJobBuilder.postCommitJob('beam_PostCommit_Py_VR_Dataflow_V2', 'Run Python Dataflow V2 ValidatesRunner',
+ 'Google Cloud Dataflow Runner V2 Python ValidatesRunner Tests', this) {
+ description('Runs Python ValidatesRunner suite on the Dataflow runner v2.')
+
+ // Set common parameters.
+ commonJobProperties.setTopLevelMainJobProperties(delegate)
+
+ publishers {
+ archiveJunit('**/nosetests*.xml')
+ }
+
+ // Execute gradle task to test Python SDK.
+ steps {
+ gradle {
+ rootBuildScriptDir(commonJobProperties.checkoutDir)
+ tasks(':sdks:python:test-suites:dataflow:py2:validatesRunnerBatchTests')
+ tasks(':sdks:python:test-suites:dataflow:py2:validatesRunnerStreamingTests')
+ // TODO: Enable following tests after making sure we have enough capacity.
Review comment:
What do we need to do for this? tracking bug?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] angoenka commented on a change in pull request #10998:
[BEAM-9287] Add Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
angoenka commented on a change in pull request #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#discussion_r385940539
##########
File path: .test-infra/jenkins/job_PostCommit_Python_ValidatesRunner_Dataflow_V2.groovy
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+import PostcommitJobBuilder
+
+// This job runs the suite of Python ValidatesRunner tests against the
+// Dataflow runner V2.
+PostcommitJobBuilder.postCommitJob('beam_PostCommit_Py_VR_Dataflow_V2', 'Run Python Dataflow V2 ValidatesRunner',
+ 'Google Cloud Dataflow Runner V2 Python ValidatesRunner Tests', this) {
+ description('Runs Python ValidatesRunner suite on the Dataflow runner v2.')
+
+ // Set common parameters.
+ commonJobProperties.setTopLevelMainJobProperties(delegate)
+
+ publishers {
+ archiveJunit('**/nosetests*.xml')
+ }
+
+ // Execute gradle task to test Python SDK.
+ steps {
+ gradle {
+ rootBuildScriptDir(commonJobProperties.checkoutDir)
+ tasks(':sdks:python:test-suites:dataflow:py2:validatesRunnerBatchTests')
+ tasks(':sdks:python:test-suites:dataflow:py2:validatesRunnerStreamingTests')
+ // TODO: Enable following tests after making sure we have enough capacity.
Review comment:
I will add a jira for it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] angoenka commented on issue #10998: [BEAM-9287] Add
Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
angoenka commented on issue #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#issuecomment-592776224
Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] angoenka commented on issue #10998: [BEAM-9287] Add
Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
angoenka commented on issue #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#issuecomment-592288401
R: @ananvay @markflyhigh
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] ananvay commented on a change in pull request #10998:
[BEAM-9287] Add Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
ananvay commented on a change in pull request #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#discussion_r385488768
##########
File path: sdks/python/scripts/run_integration_test.sh
##########
@@ -232,6 +232,7 @@ if [[ -z $PIPELINE_OPTS ]]; then
if [[ "$RUNNER_V2" = true ]]; then
opts+=("--experiments=use_runner_v2")
opts+=("--enable_streaming_engine")
+ opts+=("--experiments=runner_harness_container_image=gcr.io/cloud-dataflow/v1beta3/unified-harness:20200227-rc01")
Review comment:
should we add a todo (with a buganizer bug) to remove this once we can pick it up automatically?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] ananvay commented on issue #10998: [BEAM-9287] Add
Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
ananvay commented on issue #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#issuecomment-592749863
Thanks, LGTM!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] angoenka commented on issue #10998: [BEAM-9287] Add
Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
angoenka commented on issue #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#issuecomment-592746535
> Ah, sorry. Are the comments visible now?
Yup
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] angoenka commented on issue #10998: [BEAM-9287] Add
Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
angoenka commented on issue #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#issuecomment-592290433
Run Seed Job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] angoenka commented on issue #10998: [BEAM-9287] Add
Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
angoenka commented on issue #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#issuecomment-592289884
Run Seed Job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] angoenka commented on a change in pull request #10998:
[BEAM-9287] Add Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
angoenka commented on a change in pull request #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#discussion_r385940539
##########
File path: .test-infra/jenkins/job_PostCommit_Python_ValidatesRunner_Dataflow_V2.groovy
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+import PostcommitJobBuilder
+
+// This job runs the suite of Python ValidatesRunner tests against the
+// Dataflow runner V2.
+PostcommitJobBuilder.postCommitJob('beam_PostCommit_Py_VR_Dataflow_V2', 'Run Python Dataflow V2 ValidatesRunner',
+ 'Google Cloud Dataflow Runner V2 Python ValidatesRunner Tests', this) {
+ description('Runs Python ValidatesRunner suite on the Dataflow runner v2.')
+
+ // Set common parameters.
+ commonJobProperties.setTopLevelMainJobProperties(delegate)
+
+ publishers {
+ archiveJunit('**/nosetests*.xml')
+ }
+
+ // Execute gradle task to test Python SDK.
+ steps {
+ gradle {
+ rootBuildScriptDir(commonJobProperties.checkoutDir)
+ tasks(':sdks:python:test-suites:dataflow:py2:validatesRunnerBatchTests')
+ tasks(':sdks:python:test-suites:dataflow:py2:validatesRunnerStreamingTests')
+ // TODO: Enable following tests after making sure we have enough capacity.
Review comment:
I will add a jira for it.
However, other variant of these tests don't add much value from unified worker point of view.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] angoenka commented on issue #10998: [BEAM-9287] Add
Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
angoenka commented on issue #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#issuecomment-592691460
> Thanks a lot Ankur!!
>
> Overall LGTM, a few minor comments to track the changes made here. Perhaps we should have a bug track those internally so we don't forget to address these?
Thanks for taking a look.
I don't see any comments. Maybe they are in draft :)
github is a bit weird :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] angoenka commented on issue #10998: [BEAM-9287] Add
Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
angoenka commented on issue #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#issuecomment-592745575
A Validates runner run for dataflow v2 https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2_PR/1/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] yifanzou commented on issue #10998: [BEAM-9287] Add
Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
yifanzou commented on issue #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#issuecomment-592774593
LGTM, thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] ananvay commented on issue #10998: [BEAM-9287] Add
Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
ananvay commented on issue #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#issuecomment-592291950
Thanks a lot Ankur!!
Overall LGTM, a few minor comments to track the changes made here. Perhaps we should have a bug track those internally so we don't forget to address these?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] angoenka commented on issue #10998: [BEAM-9287] Add
Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
angoenka commented on issue #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#issuecomment-592691050
Run Seed Job
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] angoenka commented on a change in pull request #10998:
[BEAM-9287] Add Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
angoenka commented on a change in pull request #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#discussion_r385940430
##########
File path: sdks/python/scripts/run_integration_test.sh
##########
@@ -232,6 +232,7 @@ if [[ -z $PIPELINE_OPTS ]]; then
if [[ "$RUNNER_V2" = true ]]; then
opts+=("--experiments=use_runner_v2")
opts+=("--enable_streaming_engine")
+ opts+=("--experiments=runner_harness_container_image=gcr.io/cloud-dataflow/v1beta3/unified-harness:20200227-rc01")
Review comment:
yes, added with the jira
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] ananvay commented on issue #10998: [BEAM-9287] Add
Postcommit tests for dataflow runner v2
Posted by GitBox <gi...@apache.org>.
ananvay commented on issue #10998: [BEAM-9287] Add Postcommit tests for dataflow runner v2
URL: https://github.com/apache/beam/pull/10998#issuecomment-592695151
Ah, sorry. Are the comments visible now?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services