You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/04/15 16:30:40 UTC
[GitHub] [spark] HyukjinKwon opened a new pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
HyukjinKwon opened a new pull request #32193:
URL: https://github.com/apache/spark/pull/32193
### What changes were proposed in this pull request?
This PR is a POC to show the GitHub status checks instead of a comment that link to the build (from forked repository).
This is how it works:
1. "Notify test workflow" (previously created a comment) now creates a GitHub in-progress status (yellow status) to the current PR.
2. "Update build status workflow" regularly (every 15 mins) checks open PRs, and updates the GitHub status of PRs according to the status of workflows in the forked repositories.
### Why are the changes needed?
The GitHub status shows a green although the tests are running, which is confusing.
### Does this PR introduce _any_ user-facing change?
No, dev-only.
### How was this patch tested?
Manually tested at https://github.com/HyukjinKwon/spark/pull/37
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #32193:
URL: https://github.com/apache/spark/pull/32193
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-822298088
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137576/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r615160780
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
Thank you Yikun for very detailed review:)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820685924
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137426/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-821163266
**[Test build #137480 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137480/testReport)** for PR 32193 at commit [`480243c`](https://github.com/apache/spark/commit/480243cb6650e11d35fc3300e259144e476cb43a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] potiuk commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-822328005
This is cool @HyukjinKwon -> sorry, I've been a bit sick last few days, still not fully recovered but I have enough capacity to take a look and that's precisely what I planned to do as a reusable action. I will take a look at your experiences at Spark and see any potential "teething problems" and see if we can apply it to Airflow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820922091
cc @srowen and @tgravescs FYI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-822293100
**[Test build #137576 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137576/testReport)** for PR 32193 at commit [`480243c`](https://github.com/apache/spark/commit/480243cb6650e11d35fc3300e259144e476cb43a).
* This patch passes all tests.
* This patch **does not merge cleanly**.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-821093048
The latest changes were tested in https://github.com/HyukjinKwon/spark/pull/44.
This is ready for a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614911162
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
+ })
+
+ // Iterator GitHub Checks in the PR
+ for await (const cr of checkRuns.data.check_runs) {
+ if (cr.name == 'Build and test') {
+ // text contains parameters to make request in JSON.
+ const params = JSON.parse(cr.output.text)
Review comment:
Note that here it gets the status of the workflow from the forked repository that is stored in the status check of the PR open in the main repo.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820897409
**[Test build #137454 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137454/testReport)** for PR 32193 at commit [`eb47ff7`](https://github.com/apache/spark/commit/eb47ff7a5ce062a55d20f21020c26cf9db08924b).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820869903
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42029/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614884570
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
If I understand correct, the latest patch still not covers the case mentioned in https://github.com/HyukjinKwon/spark/pull/42#issuecomment-821023453 .
We need do a double check in here using [Get Pull Request](https://docs.github.com/en/rest/reference/pulls#get-a-pull-request), and check `current_pr.head.sha` == `pr.head.sha`, if not equal, we should use `current_pr.head.sha` as the ref input rather than `pr.head.sha`, or just skip current schedule status set job, let next scheduler set the status (also can avoid the status error on frequent commits case potentially).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun edited a comment on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
Yikun edited a comment on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820949006
Nice work! FYI, I do some test on https://github.com/HyukjinKwon/spark/pull/43 https://github.com/HyukjinKwon/spark/pull/42 , ~both the test triggered and results are as expected.~
The status is unexpected between 2 commits in pr, see comments in
https://github.com/HyukjinKwon/spark/pull/42#issuecomment-821002475
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614927537
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
OK, thanks for the detail explanatation, it make sense, and I also see the schedule job is less than 20s. If it still have the real problem, we will fix it when it happenes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614227854
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,54 @@
+name: Update build status
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
Review comment:
Yeah, that's a problem. I will try to integrate with ["action required" status](https://docs.github.com/en/rest/guides/getting-started-with-the-checks-api#about-check-suites) of the checks in GitHub Actions (that allows us to link URLs with some guides) to guide users to explicitly enable workflows needed.
This workflow is okay because this one is supposed to run in the main repo but if users disable the main build job, it's a problem.
FWIW, here is one example when a contributor did not enable the main build job: https://github.com/apache/spark/pull/32187
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820574816
cc @potiuk too FYI.
Let me try to describe how it works nicer tomorrow. It's a bit complicated and ugly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820566410
FYI, this PR cannot be tested in a PR. It should be merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-821117619
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42055/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820846524
**[Test build #754035632](https://github.com/HyukjinKwon/spark/actions/runs/754035632)** for PR 32193 at commit [`3a27c74`](https://github.com/HyukjinKwon/spark/commit/3a27c7440fefaf3adb7f1345095c24d0b67b5fa6).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614884570
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
If I understand correct, the latest patch still not covers the case mentioned in https://github.com/HyukjinKwon/spark/pull/42#issuecomment-821023453 .
We need do a double check using [Get Pull Request](https://docs.github.com/en/rest/reference/pulls#get-a-pull-request), and check `current_pr.head.sha` == `pr.head.sha`, if not equal, we should use `current_pr.head.sha` as the ref input rather than `pr.head.sha`, or just skip current schedule status set job, let next scheduler set the status (also can avoid the status error on frequent commits case potentially).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-821709523
Thanks Yikun for very detailed review :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-822298088
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137576/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-822329917
Thank you @potiuk! Sure health is the most important one .. !!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820867955
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42028/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614923923
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
So as long as a sha has the status check matched with the forked one (which is now guaranteed in the notify workflow), it will keep matched.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-821089656
**[Test build #137480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137480/testReport)** for PR 32193 at commit [`480243c`](https://github.com/apache/spark/commit/480243cb6650e11d35fc3300e259144e476cb43a).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820565604
**[Test build #752725108](https://github.com/HyukjinKwon/spark/actions/runs/752725108)** for PR 32193 at commit [`a7dbce3`](https://github.com/HyukjinKwon/spark/commit/a7dbce37ca7295ad3a4e1ffe1a6d2db089fa1a57).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614908174
##########
File path: .github/workflows/notify_test_workflow.yml
##########
@@ -13,28 +38,53 @@ jobs:
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
- const endpoint = "GET /repos/:owner/:repo/actions/workflows/:id/runs?&branch=:branch"
+ const endpoint = 'GET /repos/:owner/:repo/actions/workflows/:id/runs?&branch=:branch'
+
+ // TODO: Should use pull_request.user and pull_request.user.repos_url?
+ // If a different person creates a commit to another forked repo,
+ // it wouldn't be able to detect.
const params = {
owner: context.payload.pull_request.head.repo.owner.login,
repo: context.payload.pull_request.head.repo.name,
- id: "build_and_test.yml",
+ id: 'build_and_test.yml',
branch: context.payload.pull_request.head.ref,
}
+ console.log('Ref: ' + context.payload.pull_request.head.ref)
+ console.log('SHA: ' + context.payload.pull_request.head.sha)
+
+ // Wait 3 seconds to make sure the fork repository triggered a workflow.
+ await new Promise(r => setTimeout(r, 3000))
+
const runs = await github.request(endpoint, params)
- var runID = runs.data.workflow_runs[0].id
-
- var msg = "**[Test build #" + runID + "]"
- + "(https://github.com/" + context.payload.pull_request.head.repo.full_name
- + "/actions/runs/" + runID + ")** "
- + "for PR " + context.issue.number
- + " at commit [`" + context.payload.pull_request.head.sha.substring(0, 7) + "`]"
- + "(https://github.com/" + context.payload.pull_request.head.repo.full_name
- + "/commit/" + context.payload.pull_request.head.sha + ")."
-
- github.issues.createComment({
- issue_number: context.issue.number,
- owner: context.payload.repository.owner.login,
- repo: context.payload.repository.name,
- body: msg
+ const runID = runs.data.workflow_runs[0].id
+ // TODO: If no workflows were found, it's likely GitHub Actions was not enabled
+
+ if (runs.data.workflow_runs[0].head_sha != context.payload.pull_request.head.sha) {
+ throw new Error('There was a new unsynced commit pushed. Please retrigger the workflow.');
+ }
Review comment:
So I think the cause of the issue you faced is here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614923923
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
So as long as a sha has the status check matched with the forked one (which is guaranteed in the notify workflow), it will keep matched.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820601675
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614517160
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,85 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = "GET /repos/:owner/:repo/pulls?state=:state"
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ const maybeReady = ["clean", "has_hooks", "unknown", "unstable"];
+ const notReady = ["dirty", "draft"];
Review comment:
Thanks for your review @Yikun! I will take a look for that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614907754
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
@Yikun, I think it does handle. The logic here only works if the specific sha has the status check. So as long as the status check has the matched link, it will work. If it doesn't, it will skip.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-821089656
**[Test build #137480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137480/testReport)** for PR 32193 at commit [`480243c`](https://github.com/apache/spark/commit/480243cb6650e11d35fc3300e259144e476cb43a).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820867955
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42028/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820855761
**[Test build #137454 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137454/testReport)** for PR 32193 at commit [`eb47ff7`](https://github.com/apache/spark/commit/eb47ff7a5ce062a55d20f21020c26cf9db08924b).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-821919995
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614884570
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
If I understand correct, the latest patch still not covers the case mentioned in https://github.com/HyukjinKwon/spark/pull/42#issuecomment-821023453 .
We need do a double check in here using [Get Pull Request](https://docs.github.com/en/rest/reference/pulls#get-a-pull-request), and check `current_pr.head.sha` == `pr.head.sha`, if not equal, we should use `current_pr.head.sha` as the ref input rather than `pr.head.sha` in current schedule job, or just skip current schedule status set job, let next scheduler refresh the status (also can avoid the status error on frequent commits case potentially).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-821188381
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137480/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-821081190
**[Test build #755288225](https://github.com/HyukjinKwon/spark/actions/runs/755288225)** for PR 32193 at commit [`b96db67`](https://github.com/HyukjinKwon/spark/commit/b96db670e125c67e2597660a37bed84eacfc60ea).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614511024
##########
File path: .github/workflows/notify_test_workflow.yml
##########
@@ -22,19 +51,28 @@ jobs:
}
const runs = await github.request(endpoint, params)
- var runID = runs.data.workflow_runs[0].id
+ const runID = runs.data.workflow_runs[0].id
+ const runUrl = "https://github.com/"
+ + context.payload.pull_request.head.repo.full_name
+ + "/actions/runs/"
+ + runID
- var msg = "**[Test build #" + runID + "]"
- + "(https://github.com/" + context.payload.pull_request.head.repo.full_name
- + "/actions/runs/" + runID + ")** "
- + "for PR " + context.issue.number
- + " at commit [`" + context.payload.pull_request.head.sha.substring(0, 7) + "`]"
- + "(https://github.com/" + context.payload.pull_request.head.repo.full_name
- + "/commit/" + context.payload.pull_request.head.sha + ")."
+ const name = 'Build and test'
+ const head_sha = context.payload.pull_request.head.sha
+ const status = 'in_progress'
Review comment:
```suggestion
const status = 'queued'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-821115349
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42055/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614500505
##########
File path: .github/workflows/notify_test_workflow.yml
##########
@@ -14,6 +39,10 @@ jobs:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const endpoint = "GET /repos/:owner/:repo/actions/workflows/:id/runs?&branch=:branch"
+
+ // TODO: Should use pull_request.user and pull_request.user.repos_url?
+ // If a different person creates a commit to another forked repo,
+ // it would be able to detect.
Review comment:
```suggestion
// it wouldn't be able to detect.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614912919
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
> The logic here only works if the specific sha has the status check.
emm, but old (not latest) commit (multi commits in one PR) still have valid check-runs, I just a little bit confusing how to ensure the commit is the latest?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614884570
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
If I understand correct, the latest patch still not covers the case mentioned in https://github.com/HyukjinKwon/spark/pull/42#issuecomment-821023453 .
We need do a double check in here using [Get Pull Request](https://docs.github.com/en/rest/reference/pulls#get-a-pull-request), and check `current_pr.head.sha` == `pr.head.sha`, if not equal, we should use `current_pr.head.sha` as the `ref` input rather than `pr.head.sha` in current schedule job, or just skip current schedule status set job, let next scheduler refresh the status (also can avoid the status error on frequent commits case potentially).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820612592
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42003/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820892634
**[Test build #137453 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137453/testReport)** for PR 32193 at commit [`3a27c74`](https://github.com/apache/spark/commit/3a27c7440fefaf3adb7f1345095c24d0b67b5fa6).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614619818
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,85 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = "GET /repos/:owner/:repo/pulls?state=:state"
Review comment:
I tried
`curl -H "Accept: application/vnd.github.v3+json" https://api.github.com/repos/apache/spark/pulls`
and it shows 30 results.
We may need to consider adding parameters for sorting and `per_page` according to https://docs.github.com/en/rest/reference/pulls (The doc doesn't say what is the default sorting field)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820575339
**[Test build #137426 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137426/testReport)** for PR 32193 at commit [`a7dbce3`](https://github.com/apache/spark/commit/a7dbce37ca7295ad3a4e1ffe1a6d2db089fa1a57).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820612592
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42003/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614915262
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
I mean the first commit triggers the first notify check, the second trigger the frist notify check, there are two valid commits with check-runs. and then schedule job happened, the 3rd commit is coming between the schedule job.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614926181
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
In the seanrio you mentioned, the third commit will be skipped because it does not have a check run named "Build and test" set from notify workflow. If it has, the status will be updated coresspondingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820840588
**[Test build #753993713](https://github.com/HyukjinKwon/spark/actions/runs/753993713)** for PR 32193 at commit [`154e748`](https://github.com/HyukjinKwon/spark/commit/154e748245988cc8a822ef179f137c514098516b).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-822172137
**[Test build #137576 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137576/testReport)** for PR 32193 at commit [`480243c`](https://github.com/apache/spark/commit/480243cb6650e11d35fc3300e259144e476cb43a).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820901732
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614619818
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,85 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = "GET /repos/:owner/:repo/pulls?state=:state"
Review comment:
I tried
`curl -H "Accept: application/vnd.github.v3+json" https://api.github.com/repos/apache/spark/pulls`
and it shows 30 results.
We may need to consider adding parameters for sorting and `per_page`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820920543
Seems like working fine. I will:
- Address the comment https://github.com/apache/spark/pull/32193/files#r614514540
- Add some logs for debugging in the future.
Please feel free to review futher!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
Yikun commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820949006
Nice work! FYI, I do some test on https://github.com/HyukjinKwon/spark/pull/43 https://github.com/HyukjinKwon/spark/pull/42 , both the test triggered and results are as expected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820869886
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820864384
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42028/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820842879
**[Test build #754009708](https://github.com/HyukjinKwon/spark/actions/runs/754009708)** for PR 32193 at commit [`71b7949`](https://github.com/HyukjinKwon/spark/commit/71b7949f67f53ae1797a1182c1abeecb4f573cc4).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820847697
**[Test build #137453 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137453/testReport)** for PR 32193 at commit [`3a27c74`](https://github.com/apache/spark/commit/3a27c7440fefaf3adb7f1345095c24d0b67b5fa6).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614915831
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
If this specific pr.head.sha is not matched with the one from motify workflow, it will skip because the pr.head.sha wouldn't have a status check.
If pr.head.sha has the status, the sha will always be matched (although it might not be the latest). The next run of the workflow in the main repo will handle the latest one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820568682
I will show a proper example tomorrow .. it's a bit late in my time..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820685924
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137426/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-821117619
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42055/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614224749
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,54 @@
+name: Update build status
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
Review comment:
Just a question. What happens when the fork disables this job?
GitHub allows us to disable workflows. So, I usually disables the snapshot publishing jobs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820568438
Thank you, @HyukjinKwon !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614915831
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
If this specific pr.head.sha is not matched with the one from motify workflow, it will skip because the pr.head.sha wouldn't have a status check set from notify workflow.
If pr.head.sha has the status, the sha will always be matched (although it might not be the latest). The next run of the workflow in the main repo will handle the latest one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820668307
**[Test build #137426 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137426/testReport)** for PR 32193 at commit [`a7dbce3`](https://github.com/apache/spark/commit/a7dbce37ca7295ad3a4e1ffe1a6d2db089fa1a57).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614877626
##########
File path: .github/workflows/notify_test_workflow.yml
##########
@@ -13,28 +38,53 @@ jobs:
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
- const endpoint = "GET /repos/:owner/:repo/actions/workflows/:id/runs?&branch=:branch"
+ const endpoint = 'GET /repos/:owner/:repo/actions/workflows/:id/runs?&branch=:branch'
+
+ // TODO: Should use pull_request.user and pull_request.user.repos_url?
+ // If a different person creates a commit to another forked repo,
+ // it wouldn't be able to detect.
const params = {
owner: context.payload.pull_request.head.repo.owner.login,
repo: context.payload.pull_request.head.repo.name,
- id: "build_and_test.yml",
+ id: 'build_and_test.yml',
branch: context.payload.pull_request.head.ref,
}
+ console.log('Ref: ' + context.payload.pull_request.head.ref)
+ console.log('SHA: ' + context.payload.pull_request.head.sha)
+
+ // Wait 3 seconds to make sure the fork repository triggered a workflow.
+ await new Promise(r => setTimeout(r, 3000))
+
const runs = await github.request(endpoint, params)
- var runID = runs.data.workflow_runs[0].id
-
- var msg = "**[Test build #" + runID + "]"
- + "(https://github.com/" + context.payload.pull_request.head.repo.full_name
- + "/actions/runs/" + runID + ")** "
- + "for PR " + context.issue.number
- + " at commit [`" + context.payload.pull_request.head.sha.substring(0, 7) + "`]"
- + "(https://github.com/" + context.payload.pull_request.head.repo.full_name
- + "/commit/" + context.payload.pull_request.head.sha + ")."
-
- github.issues.createComment({
- issue_number: context.issue.number,
- owner: context.payload.repository.owner.login,
- repo: context.payload.repository.name,
- body: msg
+ const runID = runs.data.workflow_runs[0].id
+ // TODO: If no workflows were found, it's likely GitHub Actions was not enabled
+
+ if (runs.data.workflow_runs[0].head_sha != context.payload.pull_request.head.sha) {
+ throw new Error('There was a new unsynced commit pushed. Please retrigger the workflow.');
+ }
Review comment:
Good update, this check resolve the problme of notify_test retrigger.
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
If I understand correct, the latest patch still not cover the case mentioned in https://github.com/HyukjinKwon/spark/pull/42#issuecomment-821023453 .
We need do a double check using [Get Pull Request](https://docs.github.com/en/rest/reference/pulls#get-a-pull-request), and check `current_pr.head.sha` == `pr.head.sha`, if not equal we should use `current_pr.head.sha` as the ref input.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820847697
**[Test build #137453 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137453/testReport)** for PR 32193 at commit [`3a27c74`](https://github.com/apache/spark/commit/3a27c7440fefaf3adb7f1345095c24d0b67b5fa6).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820575339
**[Test build #137426 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137426/testReport)** for PR 32193 at commit [`a7dbce3`](https://github.com/apache/spark/commit/a7dbce37ca7295ad3a4e1ffe1a6d2db089fa1a57).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-821188381
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137480/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820901732
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614227854
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,54 @@
+name: Update build status
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
Review comment:
Yeah, that's a problem. I will try to integrate with ["action required" status](https://docs.github.com/en/rest/guides/getting-started-with-the-checks-api#about-check-suites) of the checks in GitHub Actions (that allows us to link URLs with some guides)
This workflow is okay because this one is supposed to run in the main repo but if users disable the main build job, it's a problem.
This is one example when a contributor did not enable the main build job: https://github.com/apache/spark/pull/32187
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614915262
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
I mean the first commit triggers the first notify check, the second trigger the second notify check, there are two valid commits with check-runs. and then schedule job happened, the 3rd commit is coming between the schedule job.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-821079718
**[Test build #755281266](https://github.com/HyukjinKwon/spark/actions/runs/755281266)** for PR 32193 at commit [`48ad606`](https://github.com/HyukjinKwon/spark/commit/48ad606dcff2c433621155e9d6e2a18f3048f763).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820869903
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42029/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-822172137
**[Test build #137576 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137576/testReport)** for PR 32193 at commit [`480243c`](https://github.com/apache/spark/commit/480243cb6650e11d35fc3300e259144e476cb43a).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
Yikun commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614514540
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,85 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = "GET /repos/:owner/:repo/pulls?state=:state"
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ const maybeReady = ["clean", "has_hooks", "unknown", "unstable"];
+ const notReady = ["dirty", "draft"];
Review comment:
Looks like notReady is unused, so does it just a note for reviewer? And quick question, does all `mergestatestatus` had been considered or not? looks like there are some other status except [notReady + maybeReady]
https://docs.github.com/en/graphql/reference/enums#mergestatestatus
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820855761
**[Test build #137454 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137454/testReport)** for PR 32193 at commit [`eb47ff7`](https://github.com/apache/spark/commit/eb47ff7a5ce062a55d20f21020c26cf9db08924b).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-820855152
**[Test build #754069356](https://github.com/HyukjinKwon/spark/actions/runs/754069356)** for PR 32193 at commit [`eb47ff7`](https://github.com/HyukjinKwon/spark/commit/eb47ff7a5ce062a55d20f21020c26cf9db08924b).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614915831
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
If this specific pr.head.sha is not matched with the one from motify workflow, it will skip because the pr.head.sha wouldn't have a status check.
If pr.head.sha has the status, the sha will always be matched (although it might not be the latest)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-821113131
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42055/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614926181
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
In the seanrio you mentioned, the third commit will be skipped because it does not have a check run named "Build and test"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614636082
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,85 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = "GET /repos/:owner/:repo/pulls?state=:state"
Review comment:
Oh, so it calls pagination. It will iterate batch by batch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-821781783
Will merge this in tomorrow or a day after tomorrow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-821086716
**[Test build #755317848](https://github.com/HyukjinKwon/spark/actions/runs/755317848)** for PR 32193 at commit [`480243c`](https://github.com/HyukjinKwon/spark/commit/480243cb6650e11d35fc3300e259144e476cb43a).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] potiuk commented on pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #32193:
URL: https://github.com/apache/spark/pull/32193#issuecomment-822348862
Also I commented in the builds@a.o discussion. I really think, all this should be a simple "run-in-fork" flag on GitHub Actions Workflow that you should be able to get it working "out-of-the-box". All those workarounds here would not be needed and maybe, together with the ASF we should start lobbying for GA to implement this on their side? I could write an action or two to make it easier, but still certain "brittleness" will be still there (for example the scheduled workflow to check for status of running workflows might simply sit in a queue for a long time, and running it every 15 minutes might actually cause the queues to run even longer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [DO-NOT-MERGE][POC] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614227854
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,54 @@
+name: Update build status
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
Review comment:
Yeah, that's a problem. I will try to integrate with ["action required" status](https://docs.github.com/en/rest/guides/getting-started-with-the-checks-api#about-check-suites) of the checks in GitHub Actions (that allows us to link URLs with some guides)
This workflow is okay because this one is supposed to run in the main repo but if users disable the main build job, it's a problem.
FWIW, here is one example when a contributor did not enable the main build job: https://github.com/apache/spark/pull/32187
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #32193: [SPARK-35101][INFRA] Add GitHub status check in PR instead of a comment
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #32193:
URL: https://github.com/apache/spark/pull/32193#discussion_r614915831
##########
File path: .github/workflows/update_build_status.yml
##########
@@ -0,0 +1,95 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-20.04
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v3
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks', 'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params)) {
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
Review comment:
If this specific pr.head.sha is not matched with the one from motify workflow, it will skip because the pr.head.sha wouldn't have a status.
If pr.head.sha has the status, the sha will always be matched (although it might not be the latest)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org