You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/05/20 00:29:51 UTC

[GitHub] [beam] ibzib opened a new pull request #11755: [BEAM-10038] Add script to mass-comment Jenkins triggers on PR

ibzib opened a new pull request #11755:
URL: https://github.com/apache/beam/pull/11755


   @Ardagan wrote most of this script a while back, I just generalized it a bit.
   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   
    - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
    - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
    - [ ] Update `CHANGES.md` with noteworthy changes.
    - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
   
   Post-Commit Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
   Java | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
   Python | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
   XLang | --- | --- | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
   
   Pre-Commit Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   
   --- |Java | Python | Go | Website
   --- | --- | --- | --- | ---
   Non-portable | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[![Build Status](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/) 
   Portable | --- | [![Build Status](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
   
   See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] iemejia commented on pull request #11755: [BEAM-10038] Add script to mass-comment Jenkins triggers on PR

Posted by GitBox <gi...@apache.org>.
iemejia commented on pull request #11755:
URL: https://github.com/apache/beam/pull/11755#issuecomment-632404805


   Yes we probably need those jenkins uber jobs (Flink too). Other use case where this script proves really handy would be users asking committers to trigger tests, maybe worth to announce it in the ML. Of course this is not its intended goal but realistically a more useful one until we have our own jenkins instances. Maybe we can improve it to make that task easier WDYT?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] iemejia commented on pull request #11755: [BEAM-10038] Add script to mass-comment Jenkins triggers on PR

Posted by GitBox <gi...@apache.org>.
iemejia commented on pull request #11755:
URL: https://github.com/apache/beam/pull/11755#issuecomment-631761505


   This looks pretty neat. We could maybe specialize this for some particular common cases. I found myself many times requiring  to run something like "Run All Spark Runners ValidatesRunner Tests" WDYT @ibzib ?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] ibzib commented on pull request #11755: [BEAM-10038] Add script to mass-comment Jenkins triggers on PR

Posted by GitBox <gi...@apache.org>.
ibzib commented on pull request #11755:
URL: https://github.com/apache/beam/pull/11755#issuecomment-631781814


   > > This looks pretty neat. We could maybe specialize this for some particular common cases. I found myself many times requiring to run something like "Run All Spark Runners ValidatesRunner Tests" WDYT @ibzib ?
   > 
   > If you want to use it as-is, all you have to do is modify the `COMMENTS_TO_ADD`.
   
   Actually, for that use case, maybe it would be better to make a single Jenkins uber-job that runs all the Spark test suites we have.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] robinyqiu commented on a change in pull request #11755: [BEAM-10038] Add script to mass-comment Jenkins triggers on PR

Posted by GitBox <gi...@apache.org>.
robinyqiu commented on a change in pull request #11755:
URL: https://github.com/apache/beam/pull/11755#discussion_r494686914



##########
File path: website/www/site/content/en/contribute/release-guide.md
##########
@@ -398,15 +398,14 @@ There are 2 ways to perform this verification, either running automation script(
 Jenkins job `beam_Release_Gradle_Build` basically run `./gradlew build -PisRelease`.
 This only verifies that everything builds with unit tests passing.
 
-You can refer to [this script](https://gist.github.com/Ardagan/13e6031e8d1c9ebbd3029bf365c1a517) to mass-comment on PR.
+You can use [mass_comment.py](https://github.com/apache/beam/blob/master/release/src/main/scripts/mass_comment.py) to mass-comment on PR.

Review comment:
       Oh I see the website is already updated. We should remove the reference in line 391 as well.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] ibzib commented on a change in pull request #11755: [BEAM-10038] Add script to mass-comment Jenkins triggers on PR

Posted by GitBox <gi...@apache.org>.
ibzib commented on a change in pull request #11755:
URL: https://github.com/apache/beam/pull/11755#discussion_r428096074



##########
File path: release/src/main/scripts/mass_comment.py
##########
@@ -0,0 +1,141 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""Script for mass-commenting Jenkins test triggers on a Beam PR."""
+
+import itertools
+import os
+import socket
+import sys
+import time
+import traceback
+import re
+import requests
+from datetime import datetime
+
+
+COMMENTS_TO_ADD=[
+  "Run Go PostCommit",
+  "Run Java PostCommit",
+  "Run Java PortabilityApi PostCommit",
+  "Run Java Flink PortableValidatesRunner Batch",
+  "Run Java Flink PortableValidatesRunner Streaming",
+  "Run Apex ValidatesRunner",
+  "Run Dataflow ValidatesRunner",
+  "Run Flink ValidatesRunner",
+  "Run Gearpump ValidatesRunner",
+  "Run Dataflow PortabilityApi ValidatesRunner",
+  "Run Samza ValidatesRunner",
+  "Run Spark ValidatesRunner",
+  "Run Python Dataflow ValidatesContainer",
+  "Run Python Dataflow ValidatesRunner",
+  "Run Python 3.5 Flink ValidatesRunner",
+  "Run Python 2 PostCommit",
+  "Run Python 3.5 PostCommit",
+  "Run SQL PostCommit",
+  "Run Go PreCommit",
+  "Run Java PreCommit",
+  "Run Java_Examples_Dataflow PreCommit",
+  "Run JavaPortabilityApi PreCommit",
+  "Run Portable_Python PreCommit",
+  "Run PythonLint PreCommit",
+  "Run Python PreCommit",
+  "Run Python DockerBuild PreCommit"
+]

Review comment:
       Probably we should just call mass_comment.py from verify_release_build.sh.
   
   The goal of the list is to run all the Jenkins jobs that might provide relevant signal for the release. My understanding is, since we for the most part only run Jenkins cron jobs on head, creating a dummy PR is the best way to test the release branch. I bet it's possible to configure Jenkins cron jobs to run against the release branch instead, but not sure if that would be worth the effort or expense.
   
   Also, this list does look out of date (Python 3.6 and 3.7 are conspicuously missing, for example). I'll update it later.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] ibzib commented on a change in pull request #11755: [BEAM-10038] Add script to mass-comment Jenkins triggers on PR

Posted by GitBox <gi...@apache.org>.
ibzib commented on a change in pull request #11755:
URL: https://github.com/apache/beam/pull/11755#discussion_r494688890



##########
File path: website/www/site/content/en/contribute/release-guide.md
##########
@@ -398,15 +398,14 @@ There are 2 ways to perform this verification, either running automation script(
 Jenkins job `beam_Release_Gradle_Build` basically run `./gradlew build -PisRelease`.
 This only verifies that everything builds with unit tests passing.
 
-You can refer to [this script](https://gist.github.com/Ardagan/13e6031e8d1c9ebbd3029bf365c1a517) to mass-comment on PR.
+You can use [mass_comment.py](https://github.com/apache/beam/blob/master/release/src/main/scripts/mass_comment.py) to mass-comment on PR.

Review comment:
       I missed that. Looks like it's mentioned here too:
   
   https://github.com/apache/beam/blob/0a8d955e49776b872e8fda59cbeb0870afa68784/release/src/main/scripts/verify_release_build.sh#L32
   
   Robin, can you make a PR and add me as reviewer?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] robinyqiu commented on a change in pull request #11755: [BEAM-10038] Add script to mass-comment Jenkins triggers on PR

Posted by GitBox <gi...@apache.org>.
robinyqiu commented on a change in pull request #11755:
URL: https://github.com/apache/beam/pull/11755#discussion_r494686914



##########
File path: website/www/site/content/en/contribute/release-guide.md
##########
@@ -398,15 +398,14 @@ There are 2 ways to perform this verification, either running automation script(
 Jenkins job `beam_Release_Gradle_Build` basically run `./gradlew build -PisRelease`.
 This only verifies that everything builds with unit tests passing.
 
-You can refer to [this script](https://gist.github.com/Ardagan/13e6031e8d1c9ebbd3029bf365c1a517) to mass-comment on PR.
+You can use [mass_comment.py](https://github.com/apache/beam/blob/master/release/src/main/scripts/mass_comment.py) to mass-comment on PR.

Review comment:
       Oh I see the website is already updated. We should remove the reference in line 391 as well.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] ibzib merged pull request #11755: [BEAM-10038] Add script to mass-comment Jenkins triggers on PR

Posted by GitBox <gi...@apache.org>.
ibzib merged pull request #11755:
URL: https://github.com/apache/beam/pull/11755


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] ibzib commented on a change in pull request #11755: [BEAM-10038] Add script to mass-comment Jenkins triggers on PR

Posted by GitBox <gi...@apache.org>.
ibzib commented on a change in pull request #11755:
URL: https://github.com/apache/beam/pull/11755#discussion_r428217875



##########
File path: release/src/main/scripts/mass_comment.py
##########
@@ -0,0 +1,141 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""Script for mass-commenting Jenkins test triggers on a Beam PR."""
+
+import itertools
+import os
+import socket
+import sys
+import time
+import traceback
+import re
+import requests
+from datetime import datetime
+
+
+COMMENTS_TO_ADD=[
+  "Run Go PostCommit",
+  "Run Java PostCommit",
+  "Run Java PortabilityApi PostCommit",
+  "Run Java Flink PortableValidatesRunner Batch",
+  "Run Java Flink PortableValidatesRunner Streaming",
+  "Run Apex ValidatesRunner",
+  "Run Dataflow ValidatesRunner",
+  "Run Flink ValidatesRunner",
+  "Run Gearpump ValidatesRunner",
+  "Run Dataflow PortabilityApi ValidatesRunner",
+  "Run Samza ValidatesRunner",
+  "Run Spark ValidatesRunner",
+  "Run Python Dataflow ValidatesContainer",
+  "Run Python Dataflow ValidatesRunner",
+  "Run Python 3.5 Flink ValidatesRunner",
+  "Run Python 2 PostCommit",
+  "Run Python 3.5 PostCommit",
+  "Run SQL PostCommit",
+  "Run Go PreCommit",
+  "Run Java PreCommit",
+  "Run Java_Examples_Dataflow PreCommit",
+  "Run JavaPortabilityApi PreCommit",
+  "Run Portable_Python PreCommit",
+  "Run PythonLint PreCommit",
+  "Run Python PreCommit",
+  "Run Python DockerBuild PreCommit"
+]

Review comment:
       I made some updates, PTAL




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] TheNeuralBit commented on a change in pull request #11755: [BEAM-10038] Add script to mass-comment Jenkins triggers on PR

Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on a change in pull request #11755:
URL: https://github.com/apache/beam/pull/11755#discussion_r427681057



##########
File path: release/src/main/scripts/mass_comment.py
##########
@@ -0,0 +1,141 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""Script for mass-commenting Jenkins test triggers on a Beam PR."""
+
+import itertools
+import os
+import socket
+import sys
+import time
+import traceback
+import re
+import requests
+from datetime import datetime
+
+
+COMMENTS_TO_ADD=[
+  "Run Go PostCommit",
+  "Run Java PostCommit",
+  "Run Java PortabilityApi PostCommit",
+  "Run Java Flink PortableValidatesRunner Batch",
+  "Run Java Flink PortableValidatesRunner Streaming",
+  "Run Apex ValidatesRunner",
+  "Run Dataflow ValidatesRunner",
+  "Run Flink ValidatesRunner",
+  "Run Gearpump ValidatesRunner",
+  "Run Dataflow PortabilityApi ValidatesRunner",
+  "Run Samza ValidatesRunner",
+  "Run Spark ValidatesRunner",
+  "Run Python Dataflow ValidatesContainer",
+  "Run Python Dataflow ValidatesRunner",
+  "Run Python 3.5 Flink ValidatesRunner",
+  "Run Python 2 PostCommit",
+  "Run Python 3.5 PostCommit",
+  "Run SQL PostCommit",
+  "Run Go PreCommit",
+  "Run Java PreCommit",
+  "Run Java_Examples_Dataflow PreCommit",
+  "Run JavaPortabilityApi PreCommit",
+  "Run Portable_Python PreCommit",
+  "Run PythonLint PreCommit",
+  "Run Python PreCommit",
+  "Run Python DockerBuild PreCommit"
+]

Review comment:
       Should we also remove the duplicate list from [verify_release_build.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/verify_release_build.sh#L43) as part of this PR?
   
   I'm not really clear on where this list comes from. Is the goal to launch every single jenkins job?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] TheNeuralBit commented on a change in pull request #11755: [BEAM-10038] Add script to mass-comment Jenkins triggers on PR

Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on a change in pull request #11755:
URL: https://github.com/apache/beam/pull/11755#discussion_r428148051



##########
File path: release/src/main/scripts/mass_comment.py
##########
@@ -0,0 +1,141 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""Script for mass-commenting Jenkins test triggers on a Beam PR."""
+
+import itertools
+import os
+import socket
+import sys
+import time
+import traceback
+import re
+import requests
+from datetime import datetime
+
+
+COMMENTS_TO_ADD=[
+  "Run Go PostCommit",
+  "Run Java PostCommit",
+  "Run Java PortabilityApi PostCommit",
+  "Run Java Flink PortableValidatesRunner Batch",
+  "Run Java Flink PortableValidatesRunner Streaming",
+  "Run Apex ValidatesRunner",
+  "Run Dataflow ValidatesRunner",
+  "Run Flink ValidatesRunner",
+  "Run Gearpump ValidatesRunner",
+  "Run Dataflow PortabilityApi ValidatesRunner",
+  "Run Samza ValidatesRunner",
+  "Run Spark ValidatesRunner",
+  "Run Python Dataflow ValidatesContainer",
+  "Run Python Dataflow ValidatesRunner",
+  "Run Python 3.5 Flink ValidatesRunner",
+  "Run Python 2 PostCommit",
+  "Run Python 3.5 PostCommit",
+  "Run SQL PostCommit",
+  "Run Go PreCommit",
+  "Run Java PreCommit",
+  "Run Java_Examples_Dataflow PreCommit",
+  "Run JavaPortabilityApi PreCommit",
+  "Run Portable_Python PreCommit",
+  "Run PythonLint PreCommit",
+  "Run Python PreCommit",
+  "Run Python DockerBuild PreCommit"
+]

Review comment:
       There are also some jobs that no longer exist in this list (removed in https://github.com/apache/beam/pull/11635)




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] ibzib commented on a change in pull request #11755: [BEAM-10038] Add script to mass-comment Jenkins triggers on PR

Posted by GitBox <gi...@apache.org>.
ibzib commented on a change in pull request #11755:
URL: https://github.com/apache/beam/pull/11755#discussion_r494688890



##########
File path: website/www/site/content/en/contribute/release-guide.md
##########
@@ -398,15 +398,14 @@ There are 2 ways to perform this verification, either running automation script(
 Jenkins job `beam_Release_Gradle_Build` basically run `./gradlew build -PisRelease`.
 This only verifies that everything builds with unit tests passing.
 
-You can refer to [this script](https://gist.github.com/Ardagan/13e6031e8d1c9ebbd3029bf365c1a517) to mass-comment on PR.
+You can use [mass_comment.py](https://github.com/apache/beam/blob/master/release/src/main/scripts/mass_comment.py) to mass-comment on PR.

Review comment:
       I missed that. Looks like it's mentioned here too:
   
   https://github.com/apache/beam/blob/0a8d955e49776b872e8fda59cbeb0870afa68784/release/src/main/scripts/verify_release_build.sh#L32
   
   Robin, can you make a PR and add me as reviewer?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] robinyqiu commented on pull request #11755: [BEAM-10038] Add script to mass-comment Jenkins triggers on PR

Posted by GitBox <gi...@apache.org>.
robinyqiu commented on pull request #11755:
URL: https://github.com/apache/beam/pull/11755#issuecomment-698660021


   The [release guide](https://beam.apache.org/contribute/release-guide/) on Beam website referred to the `JOB_TRIGGER_PHRASES` variable deleted in this PR. I think we should update the release guide as well.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] robinyqiu commented on pull request #11755: [BEAM-10038] Add script to mass-comment Jenkins triggers on PR

Posted by GitBox <gi...@apache.org>.
robinyqiu commented on pull request #11755:
URL: https://github.com/apache/beam/pull/11755#issuecomment-698660021


   The [release guide](https://beam.apache.org/contribute/release-guide/) on Beam website referred to the `JOB_TRIGGER_PHRASES` variable deleted in this PR. I think we should update the release guide as well.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] ibzib commented on pull request #11755: [BEAM-10038] Add script to mass-comment Jenkins triggers on PR

Posted by GitBox <gi...@apache.org>.
ibzib commented on pull request #11755:
URL: https://github.com/apache/beam/pull/11755#issuecomment-632405988


   > Yes we probably need those jenkins uber jobs (Flink too). Other use case where this script proves really handy would be users asking committers to trigger tests, maybe worth to announce it in the ML. Of course this is not its intended goal but realistically a more useful one until we have our own jenkins instances. Maybe we can improve it to make that task easier WDYT?
   
   What kind of improvements do you have in mind? It's going to be a little bit of work for the user no matter what to specify the list of jobs, so the script is only going to be useful when there are many jobs (as is the case with the release) or if they needed to be run very often for some reason (which is not common).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [beam] ibzib commented on pull request #11755: [BEAM-10038] Add script to mass-comment Jenkins triggers on PR

Posted by GitBox <gi...@apache.org>.
ibzib commented on pull request #11755:
URL: https://github.com/apache/beam/pull/11755#issuecomment-631766935


   > This looks pretty neat. We could maybe specialize this for some particular common cases. I found myself many times requiring to run something like "Run All Spark Runners ValidatesRunner Tests" WDYT @ibzib ?
   
   If you want to use it as-is, all you have to do is modify the `COMMENTS_TO_ADD`.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org