You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@yetus.apache.org by aw...@apache.org on 2020/10/12 12:53:11 UTC
[yetus] branch main updated: YETUS-1022. Get commit sha from github
PR json (#156)
This is an automated email from the ASF dual-hosted git repository.
aw pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/yetus.git
The following commit(s) were added to refs/heads/main by this push:
new 73bc655 YETUS-1022. Get commit sha from github PR json (#156)
73bc655 is described below
commit 73bc655c908f8f187b08ead5f3ac435e2e00432b
Author: Allen Wittenauer <aw...@apache.org>
AuthorDate: Mon Oct 12 05:53:01 2020 -0700
YETUS-1022. Get commit sha from github PR json (#156)
* includes some docs updates to reflect
what is supported for github
---
.../in-progress/precommit/bugsystems.html.md | 38 ++++++++++++++++++----
.../in-progress/precommit/plugins/github.html.md | 1 +
.../in-progress/precommit/robots/jenkins.html.md | 24 ++++++++++++++
precommit/src/main/shell/robots.d/jenkins.sh | 8 -----
precommit/src/main/shell/test-patch.d/github.sh | 18 ++++++++++
precommit/src/main/shell/test-patch.d/junit.sh | 1 +
6 files changed, 75 insertions(+), 15 deletions(-)
diff --git a/asf-site-src/source/documentation/in-progress/precommit/bugsystems.html.md b/asf-site-src/source/documentation/in-progress/precommit/bugsystems.html.md
index f89cc38..b19c152 100644
--- a/asf-site-src/source/documentation/in-progress/precommit/bugsystems.html.md
+++ b/asf-site-src/source/documentation/in-progress/precommit/bugsystems.html.md
@@ -67,11 +67,25 @@ Using the `--bugzilla-base-url` on the command line or BUGZILLA\_BASE\_URL in a
# GitHub Specific
-GitHub supports the full range of functionality, including putting comments on individual lines. Be aware, however, that `test-patch` will (generally) require that GitHub PRs be fully squashed and rebased (i.e., a single commit) in many circumstances.
-
-By default, the GitHub plug-in assumes that <https://github.com> is the base URL for GitHub. Enterprise users may override this with the `--github-base-url` for the normal web user interface and `--github-api-url` for the API URL. Personalities may use GITHUB\_API\_URL and GITHUB\_BASE\_URL.
-
-The specific repository on GitHub is defined with either `--github-repo` on the command line or GITHUB\_REPO in a personality. It should take the form of "user/repo".
+GitHub supports a large range of functionality, mostly self-adjusting based upon the
+environment it is running in. Supported features include GitHub Enterprise, private
+repositories, GitHub Statuses, and GitHub Checks Annotations.
+
+Be aware, however, that `test-patch` will (generally) require
+that GitHub PRs be fully squashed and rebased (i.e., a single commit) in many
+circumstances, especially renames followed by commits to those renamed files. It will
+try to fall back to 'diff' format if 'patch' format does not work, exiting
+completely if both fail.
+
+By default, the GitHub plug-in assumes that <https://github.com> is the base URL
+for GitHub and <https://api.github.com> for base of the GitHub API URL. Enterprise
+users may override this with the `--github-base-url` for the normal web user
+interface and `--github-api-url` for the API URL. Personalities may use
+GITHUB\_API\_URL and GITHUB\_BASE\_URL.
+
+The specific repository on GitHub is defined with either `--github-repo` on the
+command line, GITHUB\_REPO in a personality, or automatically determined by various
+robot support mechanisms. It should take the form of "user/repo".
GitHub pull requests may be directly processed on the command line in two ways:
@@ -83,13 +97,23 @@ Pull requests that are made off of a specific branch will switch the test repo t
## GitHub Authentication
-In order to comment on issues or, depending upon the security setup of the repo, authentication credentials. The GitHub plug-in supports authentication via token or user name/passphrase.
+Some operations depend upon authentication credentials. The GitHub plug-in
+supports authentication via token or user name/passphrase. Authentication is
+required for controlling the API query rate, accessing private repositories,
+reading private pull requests, updating GitHub Statuses (requires
+`repo:status` permissions), writing comments to PRs, and other functionality.
+
+Some [robots](../robots) have built-in support for providing GitHub tokens. See
+the appropriate documentation for your particular setup.
### GitHub Token
The token is provided via the `--github-token` option.
-### GitHub Username/password (Deprecated)
+### GitHub Username/password
+
+ NOTE: GitHub is deprecating this format. It is kept here for backward compatibility
+ until such a time the format is completely removed from modern implementations.
The user name is provided via the `--github-user` option or the GITHUB\_USER environment variable. The default value for GITHUB\_USER is the value of `--project` suffixed with QA. For example,
`--project=yetus` will set `GITHUB_USER=yetusqa`.
diff --git a/asf-site-src/source/documentation/in-progress/precommit/plugins/github.html.md b/asf-site-src/source/documentation/in-progress/precommit/plugins/github.html.md
index 8b21172..f22ab96 100644
--- a/asf-site-src/source/documentation/in-progress/precommit/plugins/github.html.md
+++ b/asf-site-src/source/documentation/in-progress/precommit/plugins/github.html.md
@@ -42,6 +42,7 @@ None
| `--github-password=<pw>` | Password or OAuth 2.0 token to use for authentication |
| `--github-repo=<repo>` | `username/repository` identifier |
| `--github-user=<user>` | Username to use for authentication |
+| `--github-token=<token>` | Token used to perform read and write operations |
# Docker Notes
diff --git a/asf-site-src/source/documentation/in-progress/precommit/robots/jenkins.html.md b/asf-site-src/source/documentation/in-progress/precommit/robots/jenkins.html.md
index 8303808..4d81947 100644
--- a/asf-site-src/source/documentation/in-progress/precommit/robots/jenkins.html.md
+++ b/asf-site-src/source/documentation/in-progress/precommit/robots/jenkins.html.md
@@ -71,6 +71,28 @@ Experience has shown that certain Jenkins + Java + OS combinations have problems
}
```
+## GitHub Token Support
+
+Using the Jenkins Credential system, one can provide a specific personal access token
+to use with GitHub. However, it is recommended that Jenkins be configured to act as
+a GitHub application as per the
+[Cloudbees documentation](https://docs.cloudbees.com/docs/cloudbees-jenkins-platform/latest/github-app-auth)
+for the optimal `test-patch` experience. Configure up to the "Configuring the GitHub Organization" and then, using the Jenkins credential system, pass the GitHub App's token to `test-patch`. For example:
+
+```groovy
+...
+ withCredentials([usernamePassword(credentialsId: 'github-app',
+ passwordVariable: 'GITHUB_TOKEN',
+ usernameVariable: 'GITHUB_USER')]) {
+...
+
+ sh '''test-patch --github-token="${GITHUB_TOKEN}" (other options)'''
+...
+```
+
+Doing so will enable in many circumstances a bit more functionality, such as
+GitHub Statuses.
+
See also:
* Apache Yetus' source tree [Jenkinsfile](https://github.com/apache/yetus/blob/main/Jenkinsfile) for some tips and tricks.
@@ -78,3 +100,5 @@ See also:
* [GitHub Branch Source Plugin](https://wiki.jenkins.io/display/JENKINS/GitHub+Branch+Source+Plugin)
* [GitHub Pull Request Builder Plugin](https://wiki.jenkins.io/display/JENKINS/GitHub+pull+request+builder+plugin)
* `https://{your local server}/env-vars.html/`
+* [From Jenkins – GitHub App authentication support released](https://cd.foundation/blog/2020/04/22/from-jenkins-github-app-authentication-support-released/)
+* [Jenkins - Using GitHub App authentication](https://docs.cloudbees.com/docs/cloudbees-jenkins-platform/latest/github-app-auth)
diff --git a/precommit/src/main/shell/robots.d/jenkins.sh b/precommit/src/main/shell/robots.d/jenkins.sh
index 789df13..8a4c6b4 100755
--- a/precommit/src/main/shell/robots.d/jenkins.sh
+++ b/precommit/src/main/shell/robots.d/jenkins.sh
@@ -99,17 +99,9 @@ if [[ -n "${JENKINS_URL}" && -n "${EXECUTOR_NUMBER}" ]] &&
EXECUTOR_NUMBER \
ghprbPullId \
ghprbPullLink \
- GIT_COMMIT \
GIT_URL \
JENKINS_URL
- # Jenkins doesn't always define this, so this
- # dependency is highly unreliable
- if [[ -n "${GIT_COMMIT}" ]]; then
- # shellcheck disable=SC2034
- GIT_BRANCH_SHA=${GIT_COMMIT}
- fi
-
yetus_add_array_element EXEC_MODES Jenkins
fi
diff --git a/precommit/src/main/shell/test-patch.d/github.sh b/precommit/src/main/shell/test-patch.d/github.sh
index 7046fd9..4e01a5d 100755
--- a/precommit/src/main/shell/test-patch.d/github.sh
+++ b/precommit/src/main/shell/test-patch.d/github.sh
@@ -304,6 +304,9 @@ function github_locate_pr_patch
declare patchout=$2
declare diffout=$3
declare apiurl
+ declare line
+ declare sha
+ declare foundhead=false
input=${input#GH:}
@@ -381,6 +384,21 @@ function github_locate_pr_patch
return 1
fi
+ if [[ -z "${GIT_BRANCH_SHA}" ]]; then
+ while read -r line; do
+ if [[ "${line}" =~ \"head\": ]]; then
+ foundhead=true
+ fi
+ if [[ "${foundhead}" == true ]]; then
+ if [[ "${line}" =~ \"sha\": ]]; then
+ sha=${line##* \"}
+ GIT_BRANCH_SHA=${sha%%\"*}
+ break
+ fi
+ fi
+ done < <(cat "${PATCH_DIR}/github-pull.json")
+ fi
+
GITHUB_ISSUE=${input}
# github will translate this to be #(xx) !
diff --git a/precommit/src/main/shell/test-patch.d/junit.sh b/precommit/src/main/shell/test-patch.d/junit.sh
index eff1f51..af0c7d1 100755
--- a/precommit/src/main/shell/test-patch.d/junit.sh
+++ b/precommit/src/main/shell/test-patch.d/junit.sh
@@ -157,6 +157,7 @@ function junit_finalreport
url=$(get_artifact_url)
cat << EOF > "${JUNIT_REPORT_XML}"
+<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite tests="1" failures="'${result}'" time="1" name="Apache Yetus">
EOF