You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficcontrol.apache.org by GitBox <gi...@apache.org> on 2021/02/09 14:07:00 UTC
[GitHub] [trafficcontrol] zrhoffman opened a new pull request #5506: Workflow to check for and PR minor Go revision updates
zrhoffman opened a new pull request #5506:
URL: https://github.com/apache/trafficcontrol/pull/5506
<!--
************ STOP!! ************
If this Pull Request is intended to fix a security vulnerability, DO NOT submit it! Instead, contact
the Apache Software Foundation Security Team at security@trafficcontrol.apache.org and follow the
guidelines at https://www.apache.org/security/ regarding vulnerability disclosure.
-->
## What does this PR (Pull Request) do?
<!-- Explain the changes you made here. If this fixes an Issue, identify it by
replacing the text in the checkbox item with the Issue number e.g.
- [x] This PR fixes #9001 OR is not related to any Issue
^ This will automatically close Issue number 9001 when the Pull Request is
merged (The '#' is important).
Be sure you check the box properly, see the "The following criteria are ALL
met by this PR" section for details.
-->
This PR adds a GitHub Actions workflow that checks once a day if a newer minor revision exists for our Go major version (currently `1.15`). If so, the workflow will
- Create a new branch in the *apache/trafficcontrol* repo and update the `GO_VERSION` file on that branch
- Open a pull request from that branch to *apache/trafficcontrol*'s `master` branch.
Note that the workflow will *not* create a pull request for major version updates like `1.15.7` -> `1.16.0`.
- [x] This PR is not related to any Issue <!-- You can check for an issue here: https://github.com/apache/trafficcontrol/issues -->
## Which Traffic Control components are affected by this PR?
<!-- Please delete all components from this list that are NOT affected by this
Pull Request. Also, feel free to add the name of a tool or script that is
affected but not on the list.
Additionally, if this Pull Request does NOT affect documentation, please
explain why documentation is not required. -->
- CI tests
## What is the best way to verify this PR?
<!-- Please include here ALL the steps necessary to test your Pull Request. If
it includes tests (and most should), outline here the steps needed to run the
tests. If not, lay out the manual testing procedure and please explain why
tests are unnecessary for this Pull Request. -->
1. Test the workflow
1. Push the `go-versions-prs` branch to your own fork
2. Commit and push the following changes:
```patch
diff --git a/.github/workflows/pr-to-update-go.yml b/.github/workflows/pr-to-update-go.yml
index f62ccd0da..8e281ac2f 100644
--- a/.github/workflows/pr-to-update-go.yml
+++ b/.github/workflows/pr-to-update-go.yml
@@ -18,8 +18,7 @@
name: Is time to Update Go?
on:
- schedule:
- - cron: 0 14 * * *
+ push:
jobs:
pr-to-update-go:
@@ -27,7 +26,6 @@ jobs:
steps:
- name: Checkout repo
uses: actions/checkout@master
- if: ${{ github.repository_owner == 'apache' }}
id: checkout
- name: Install Python 3.9
uses: actions/setup-python@v2
```
3. Verify that the pull request created in your fork contain the changes necessary to update the Go version for your fork.
2. Run the tests:
```shell
cd .github/actions/pr-to-update-go
python3 -m unittest discover ./tests
```
## The following criteria are ALL met by this PR
<!-- Check the boxes to signify that the associated statement is true. To
"check a box", replace the space inside of the square brackets with an 'x'.
e.g.
- [ x] <- Wrong
- [x ] <- Wrong
- [] <- Wrong
- [*] <- Wrong
- [x] <- Correct!
-->
- [x] This PR includes tests
- [x] This PR includes documentation
- [x] An update to CHANGELOG.md is not necessary
- [x] This PR includes any and all required license headers
- [x] This PR **DOES NOT FIX A SERIOUS SECURITY VULNERABILITY** (see [the Apache Software Foundation's security guidelines](https://www.apache.org/security/) for details)
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #5506: Workflow to check for and PR minor Go revision updates
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on a change in pull request #5506:
URL: https://github.com/apache/trafficcontrol/pull/5506#discussion_r574052640
##########
File path: .github/actions/pr-to-update-go/pr_to_update_go/__main__.py
##########
@@ -0,0 +1,32 @@
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+import os
+import sys
+
+from github.MainClass import Github
+
+from pr_to_update_go.GoPRMaker import GoPRMaker
+from pr_to_update_go.constants import ENV_GITHUB_TOKEN
+
+
+def main() -> None:
+ try:
+ github_token: str = os.environ[ENV_GITHUB_TOKEN]
+ except KeyError:
+ print(f'Environment variable {ENV_GITHUB_TOKEN} must be defined.')
+ sys.exit(1)
+ gh = Github(login_or_token=github_token)
+ GoPRMaker(gh).run()
+
+
+main()
Review comment:
If the filename is `__main__.py`, then `__name__` is always equal to `'__main__'`.
##########
File path: .github/actions/pr-to-update-go/pr_to_update_go/pr_template.md
##########
@@ -0,0 +1,120 @@
+<!--
+************ STOP!! ************
+If this Pull Request is intended to fix a security vulnerability, DO NOT submit it! Instead, contact
+the Apache Software Foundation Security Team at security@trafficcontrol.apache.org and follow the
+guidelines at https://www.apache.org/security/ regarding vulnerability disclosure.
+-->
+## What does this PR (Pull Request) do?
+<!-- Explain the changes you made here. If this fixes an Issue, identify it by
+replacing the text in the checkbox item with the Issue number e.g.
+
+- [x] This PR fixes #9001 OR is not related to any Issue
+
+^ This will automatically close Issue number 9001 when the Pull Request is
+merged (The '#' is important).
+
+Be sure you check the box properly, see the "The following criteria are ALL
+met by this PR" section for details.
+-->
Review comment:
Removed comments in de4b55e9f3
##########
File path: .github/actions/pr-to-update-go/pr_to_update_go/GoPRMaker.py
##########
@@ -0,0 +1,210 @@
+#!/usr/bin/env python3
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+import json
+import os
+import re
+import requests
+import sys
+
+from github.InputGitAuthor import InputGitAuthor
+from github.Label import Label
+from requests import Response
+
+from github.Branch import Branch
+from github.GithubException import BadCredentialsException, GithubException, UnknownObjectException
+from github.MainClass import Github
+from github.Milestone import Milestone
+from github.PaginatedList import PaginatedList
+from github.PullRequest import PullRequest
+from github.Repository import Repository
+
+from pr_to_update_go.constants import ENV_GITHUB_TOKEN, GO_VERSION_URL, ENV_GITHUB_REPOSITORY, \
+ ENV_GITHUB_REPOSITORY_OWNER, GO_REPO_NAME, RELEASE_PAGE_URL, ENV_GO_VERSION_FILE, ENV_GIT_AUTHOR_NAME, \
+ GIT_AUTHOR_EMAIL_TEMPLATE
+
+
+class GoPRMaker:
+ gh: Github
+ latest_go_version: str
+ repo: Repository
+
+ def __init__(self, gh: Github) -> None:
+ self.gh = gh
+ repo_name: str = self.get_repo_name()
+ self.repo = self.get_repo(repo_name)
+
+ def run(self) -> None:
Review comment:
I like `-> None:` because then PyCharm gives you warnings when trying to a value returned by a function whose return type is `None` without needing to analyze control flow. For example, on line 6 of this snippet,
```python
def a() -> None:
if False:
return 1
print('Returns nothing')
b: int = a()
```
it warns
> Expected type 'int', got 'None' instead.
But that warning is gone if `-> None` is removed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #5506: Workflow to check for and PR minor Go revision updates
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on a change in pull request #5506:
URL: https://github.com/apache/trafficcontrol/pull/5506#discussion_r577167937
##########
File path: .github/actions/pr-to-update-go/pr_to_update_go/go_pr_maker.py
##########
@@ -0,0 +1,391 @@
+#!/usr/bin/env python3
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+"""
+Generate pull requests that update a repository's Go version.
+
+Classes:
+
+ GoPRMaker
+
+"""
+import json
+import os
+import re
+import subprocess
+import sys
+from typing import Union
+
+import requests
+from github.Branch import Branch
+from github.Commit import Commit
+from github.GitCommit import GitCommit
+from github.GitRef import GitRef
+from github.GitTree import GitTree
+from github.GithubObject import NotSet
+from github.InputGitTreeElement import InputGitTreeElement
+from github.Requester import Requester
+
+from requests import Response
+
+from github.GithubException import BadCredentialsException, GithubException, UnknownObjectException
+from github.InputGitAuthor import InputGitAuthor
+from github.Label import Label
+from github.MainClass import Github
+from github.Milestone import Milestone
+from github.PaginatedList import PaginatedList
+from github.PullRequest import PullRequest
+from github.Repository import Repository
+
+from pr_to_update_go.constants import ENV_GITHUB_TOKEN, GO_VERSION_URL, ENV_GITHUB_REPOSITORY, \
+ ENV_GITHUB_REPOSITORY_OWNER, GO_REPO_NAME, RELEASE_PAGE_URL, ENV_GO_VERSION_FILE, \
+ ENV_GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL_TEMPLATE
+
+
+class GoPRMaker:
+ """
+ A class to generate pull requests for the purpose of updating the Go version in a repository.
+ """
+ gh: Github
+ latest_go_version: str
+ repo: Repository
+ author: InputGitAuthor
+
+ def __init__(self, gh: Github) -> None:
+ """
+ :param gh: Github
+ :rtype: None
+ """
+ self.gh = gh
+ repo_name: str = self.get_repo_name()
+ self.repo = self.get_repo(repo_name)
+
+ try:
+ git_author_name = self.getenv(ENV_GIT_AUTHOR_NAME)
+ git_author_email = GIT_AUTHOR_EMAIL_TEMPLATE.format(git_author_name=git_author_name)
+ self.author = InputGitAuthor(git_author_name, git_author_email)
+ except KeyError:
+ self.author = NotSet
+ print('Will commit using the default author')
+
+ def branch_exists(self, branch: str) -> bool:
+ """
+ :param branch:
+ :type branch:
+ :return:
+ :rtype: bool
+ """
+ try:
+ repo_go_version = self.get_repo_go_version(branch)
+ if self.latest_go_version == repo_go_version:
+ print(f'Branch {branch} already exists')
+ return True
+ except GithubException as e:
+ message = e.data.get('message')
+ if not re.match(r'No commit found for the ref', message):
+ raise e
+ return False
+
+ def update_branch(self, branch_name: str, sha: str) -> None:
+ """
+ :param branch_name:
+ :type branch_name:
+ :param sha:
+ :type sha:
+ :return:
+ :rtype: None
+ """
+ requester: Requester = self.repo._requester
+ patch_parameters = {
+ 'sha': sha,
+ }
+ requester.requestJsonAndCheck(
+ 'PATCH', self.repo.url + f'/git/refs/heads/{branch_name}', input=patch_parameters
+ )
+ return
+
+ def run(self) -> None:
+ """
+ :return:
+ :rtype: None
+ """
+ repo_go_version = self.get_repo_go_version()
+ self.latest_go_version = self.get_latest_major_upgrade(repo_go_version)
+ commit_message: str = f'Update Go version to {self.latest_go_version}'
+
+ source_branch_name: str = f'go-{self.latest_go_version}'
+ target_branch: str = 'master'
+ if repo_go_version == self.latest_go_version:
+ print(f'Go version is up-to-date on {target_branch}, nothing to do.')
+ return
+
+ if not self.branch_exists(source_branch_name):
+ commit: Commit = self.set_go_version(self.latest_go_version, commit_message,
+ source_branch_name)
+ update_golang_org_x_commit: Union[GitCommit, None] = self.update_golang_org_x(commit)
+ if isinstance(update_golang_org_x_commit, GitCommit):
+ sha: str = update_golang_org_x_commit.sha
+ self.update_branch(source_branch_name, sha)
+
+ owner: str = self.get_repo_owner()
+ self.create_pr(self.latest_go_version, commit_message, owner, source_branch_name,
+ target_branch)
+
+ @staticmethod
+ def getenv(env_name: str) -> str:
+ """
+ :param env_name: str
+ :return:
+ :rtype: str
+ """
+ return os.environ[env_name]
+
+ def get_repo(self, repo_name: str) -> Repository:
+ """
+ :param repo_name: str
+ :return:
+ :rtype: Repository
+ """
+ try:
+ repo: Repository = self.gh.get_repo(repo_name)
+ except BadCredentialsException:
+ print(f'Credentials from {ENV_GITHUB_TOKEN} were bad.')
+ sys.exit(1)
+ return repo
+
+ @staticmethod
+ def get_major_version(from_go_version: str) -> str:
+ """
+ :param from_go_version: str
+ :return:
+ :rtype: str
+ """
+ return re.search(pattern=r'^\d+\.\d+', string=from_go_version).group(0)
+
+ def get_latest_major_upgrade(self, from_go_version: str) -> str:
+ """
+ :param from_go_version: str
+ :return:
+ :rtype: str
+ """
+ major_version = self.get_major_version(from_go_version)
+ go_version_response: Response = requests.get(GO_VERSION_URL)
+ go_version_response.raise_for_status()
+ go_version_content: list = json.loads(go_version_response.content)
+ index = 0
+ fetched_go_version: str = ''
+ while True:
+ if not go_version_content[index]['stable']:
+ continue
+ go_version_name: str = go_version_content[index]['version']
+ fetched_go_version = re.search(pattern=r'[\d.]+', string=go_version_name).group(0)
+ if major_version == self.get_major_version(fetched_go_version):
+ break
+ index += 1
+ if major_version != self.get_major_version(fetched_go_version):
+ raise Exception(f'No supported {major_version} Go versions exist.')
+ print(f'Latest version of Go {major_version} is {fetched_go_version}')
+ return fetched_go_version
+
+ def get_repo_name(self) -> str:
+ """
+ :return:
+ :rtype: str
+ """
+ repo_name: str = self.getenv(ENV_GITHUB_REPOSITORY)
+ return repo_name
+
+ def get_repo_owner(self) -> str:
+ """
+ :return:
+ :rtype: str
+ """
+ repo_name: str = self.getenv(ENV_GITHUB_REPOSITORY_OWNER)
+ return repo_name
+
+ def get_go_milestone(self, go_version: str) -> str:
+ """
+ :param go_version: str
+ :return:
+ """
+ go_repo: Repository = self.get_repo(GO_REPO_NAME)
+ milestones: PaginatedList[Milestone] = go_repo.get_milestones(state='all', sort='due_on',
+ direction='desc')
+ milestone_title = f'Go{go_version}'
+ for milestone in milestones: # type: Milestone
+ if milestone.title == milestone_title:
+ print(f'Found Go milestone {milestone.title}')
+ return milestone.raw_data.get('html_url')
+ raise Exception(f'Could not find a milestone named {milestone_title}.')
+
+ @staticmethod
+ def get_release_notes_page() -> str:
+ """
+ :return:
+ :rtype: str
+ """
+ release_history_response: Response = requests.get(RELEASE_PAGE_URL)
+ release_history_response.raise_for_status()
+ return release_history_response.content.decode()
+
+ @staticmethod
+ def get_release_notes(go_version: str, release_notes_content: str) -> str:
+ """
+ :param go_version: str
+ :param release_notes_content: str
+ :return:
+ :rtype: str
+ """
+ go_version_pattern = go_version.replace('.', '\\.')
+ release_notes_pattern: str = f'<p>\\s*\\n\\s*go{go_version_pattern}.*?</p>'
+ release_notes_matches = re.search(release_notes_pattern, release_notes_content,
+ re.MULTILINE | re.DOTALL)
+ if release_notes_matches is None:
+ raise Exception(f'Could not find release notes on {RELEASE_PAGE_URL}')
+ release_notes = re.sub(r'[\s\t]+', ' ', release_notes_matches.group(0))
+ return release_notes
+
+ def get_pr_body(self, go_version: str, milestone_url: str) -> str:
+ """
+ :param go_version: str
+ :param milestone_url: str
+ :return:
+ :rtype: str
+ """
+ with open(os.path.dirname(__file__) + '/pr_template.md') as file:
+ pr_template = file.read()
+ go_major_version = self.get_major_version(go_version)
+
+ release_notes = self.get_release_notes(go_version, self.get_release_notes_page())
+ pr_body: str = pr_template.format(GO_VERSION=go_version, GO_MAJOR_VERSION=go_major_version,
+ RELEASE_NOTES=release_notes, MILESTONE_URL=milestone_url)
+ print('Templated PR body')
+ return pr_body
+
+ def get_repo_go_version(self, branch: str = 'master') -> str:
+ """
+ :param branch: str
+ :return:
+ :rtype: str
+ """
+ return self.repo.get_contents(self.getenv(ENV_GO_VERSION_FILE),
+ f'refs/heads/{branch}').decoded_content.rstrip().decode()
+
+ def set_go_version(self, go_version: str, commit_message: str,
+ source_branch_name: str) -> Commit:
+ """
+ :param go_version: str
+ :param commit_message: str
+ :param source_branch_name: str
+ :return:
+ :rtype: str
+ """
+ master: Branch = self.repo.get_branch('master')
+ sha: str = master.commit.sha
+ ref: str = f'refs/heads/{source_branch_name}'
+ self.repo.create_git_ref(ref, sha)
+
+ print(f'Created branch {source_branch_name}')
+ go_version_file: str = self.getenv(ENV_GO_VERSION_FILE)
+ go_file_contents = self.repo.get_contents(go_version_file, ref)
+ kwargs = {'path': go_version_file,
+ 'message': commit_message,
+ 'content': (go_version + '\n'),
+ 'sha': go_file_contents.sha,
+ 'branch': source_branch_name,
+ }
+ try:
+ git_author_name = self.getenv(ENV_GIT_AUTHOR_NAME)
+ git_author_email = GIT_AUTHOR_EMAIL_TEMPLATE.format(git_author_name=git_author_name)
+ author: InputGitAuthor = InputGitAuthor(name=git_author_name, email=git_author_email)
+ kwargs['author'] = author
+ kwargs['committer'] = author
+ except KeyError:
+ print('Committing using the default author')
+
+ commit: Commit = self.repo.update_file(**kwargs).get('commit')
+ print(f'Updated {go_version_file} on {self.repo.name}')
+ return commit
+
+ def update_golang_org_x(self, previous_commit: Commit) -> Union[GitCommit, None]:
+ """
+ :param previous_commit:
+ :type previous_commit:
+ :return:
+ :rtype: Union[GitCommit, None]
+ """
+ subprocess.run(['git', 'fetch', 'origin'], check=True)
+ subprocess.run(['git', 'checkout', previous_commit.sha], check=True)
+ script_path: str = f'.github/actions/pr-to-update-go/update_golang_org_x.sh'
+ subprocess.run([script_path], check=True)
+ files_to_check: list[str] = ['go.mod', 'go.sum', 'vendor/modules.txt']
+ tree_elements: list[InputGitTreeElement] = []
+ for file in files_to_check:
+ diff_process = subprocess.run(['git', 'diff', '--exit-code', '--', file])
+ if diff_process.returncode == 0:
+ continue
+ with open(file) as stream:
+ content: str = stream.read()
+ tree_element: InputGitTreeElement = InputGitTreeElement(path=file, mode='100644',
+ type='blob', content=content)
+ tree_elements.append(tree_element)
+ if len(tree_elements) == 0:
+ print('No golang.org/x/ dependencies need to be updated.')
+ return
+ tree_hash = subprocess.check_output(
+ ['git', 'log', '-1', '--pretty=%T', previous_commit.sha]).decode().strip()
+ base_tree: GitTree = self.repo.get_git_tree(sha=tree_hash)
+ tree: GitTree = self.repo.create_git_tree(tree_elements, base_tree)
+ commit_message: str = f'Update golang.org/x/ dependencies for go{self.latest_go_version}'
+ previous_git_commit: GitCommit = self.repo.get_git_commit(previous_commit.sha)
+ git_commit: GitCommit = self.repo.create_git_commit(message=commit_message, tree=tree,
+ parents=[previous_git_commit],
+ author=self.author, committer=self.author)
+ print(f'Updated golang.org/x/ dependencies')
Review comment:
Removed *f* in 978e5f94cd
##########
File path: .github/actions/pr-to-update-go/pr_to_update_go/go_pr_maker.py
##########
@@ -0,0 +1,391 @@
+#!/usr/bin/env python3
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+"""
+Generate pull requests that update a repository's Go version.
+
+Classes:
+
+ GoPRMaker
+
+"""
+import json
+import os
+import re
+import subprocess
+import sys
+from typing import Union
+
+import requests
+from github.Branch import Branch
+from github.Commit import Commit
+from github.GitCommit import GitCommit
+from github.GitRef import GitRef
Review comment:
Removed unused import in b8a2a9e6eb
##########
File path: .github/actions/pr-to-update-go/pr_to_update_go/go_pr_maker.py
##########
@@ -0,0 +1,391 @@
+#!/usr/bin/env python3
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+"""
+Generate pull requests that update a repository's Go version.
+
+Classes:
+
+ GoPRMaker
+
+"""
+import json
+import os
+import re
+import subprocess
+import sys
+from typing import Union
+
+import requests
+from github.Branch import Branch
+from github.Commit import Commit
+from github.GitCommit import GitCommit
+from github.GitRef import GitRef
+from github.GitTree import GitTree
+from github.GithubObject import NotSet
+from github.InputGitTreeElement import InputGitTreeElement
+from github.Requester import Requester
+
+from requests import Response
+
+from github.GithubException import BadCredentialsException, GithubException, UnknownObjectException
+from github.InputGitAuthor import InputGitAuthor
+from github.Label import Label
+from github.MainClass import Github
+from github.Milestone import Milestone
+from github.PaginatedList import PaginatedList
+from github.PullRequest import PullRequest
+from github.Repository import Repository
+
+from pr_to_update_go.constants import ENV_GITHUB_TOKEN, GO_VERSION_URL, ENV_GITHUB_REPOSITORY, \
+ ENV_GITHUB_REPOSITORY_OWNER, GO_REPO_NAME, RELEASE_PAGE_URL, ENV_GO_VERSION_FILE, \
+ ENV_GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL_TEMPLATE
+
+
+class GoPRMaker:
+ """
+ A class to generate pull requests for the purpose of updating the Go version in a repository.
+ """
+ gh: Github
+ latest_go_version: str
+ repo: Repository
+ author: InputGitAuthor
+
+ def __init__(self, gh: Github) -> None:
+ """
+ :param gh: Github
+ :rtype: None
+ """
+ self.gh = gh
+ repo_name: str = self.get_repo_name()
+ self.repo = self.get_repo(repo_name)
+
+ try:
+ git_author_name = self.getenv(ENV_GIT_AUTHOR_NAME)
+ git_author_email = GIT_AUTHOR_EMAIL_TEMPLATE.format(git_author_name=git_author_name)
+ self.author = InputGitAuthor(git_author_name, git_author_email)
+ except KeyError:
+ self.author = NotSet
+ print('Will commit using the default author')
+
+ def branch_exists(self, branch: str) -> bool:
+ """
+ :param branch:
+ :type branch:
+ :return:
+ :rtype: bool
+ """
+ try:
+ repo_go_version = self.get_repo_go_version(branch)
+ if self.latest_go_version == repo_go_version:
+ print(f'Branch {branch} already exists')
+ return True
+ except GithubException as e:
+ message = e.data.get('message')
+ if not re.match(r'No commit found for the ref', message):
+ raise e
+ return False
+
+ def update_branch(self, branch_name: str, sha: str) -> None:
+ """
+ :param branch_name:
+ :type branch_name:
+ :param sha:
+ :type sha:
+ :return:
+ :rtype: None
+ """
+ requester: Requester = self.repo._requester
+ patch_parameters = {
+ 'sha': sha,
+ }
+ requester.requestJsonAndCheck(
+ 'PATCH', self.repo.url + f'/git/refs/heads/{branch_name}', input=patch_parameters
+ )
+ return
+
+ def run(self) -> None:
+ """
+ :return:
+ :rtype: None
+ """
+ repo_go_version = self.get_repo_go_version()
+ self.latest_go_version = self.get_latest_major_upgrade(repo_go_version)
+ commit_message: str = f'Update Go version to {self.latest_go_version}'
+
+ source_branch_name: str = f'go-{self.latest_go_version}'
+ target_branch: str = 'master'
+ if repo_go_version == self.latest_go_version:
+ print(f'Go version is up-to-date on {target_branch}, nothing to do.')
+ return
+
+ if not self.branch_exists(source_branch_name):
+ commit: Commit = self.set_go_version(self.latest_go_version, commit_message,
+ source_branch_name)
+ update_golang_org_x_commit: Union[GitCommit, None] = self.update_golang_org_x(commit)
+ if isinstance(update_golang_org_x_commit, GitCommit):
+ sha: str = update_golang_org_x_commit.sha
+ self.update_branch(source_branch_name, sha)
+
+ owner: str = self.get_repo_owner()
+ self.create_pr(self.latest_go_version, commit_message, owner, source_branch_name,
+ target_branch)
+
+ @staticmethod
+ def getenv(env_name: str) -> str:
+ """
+ :param env_name: str
+ :return:
+ :rtype: str
+ """
+ return os.environ[env_name]
+
+ def get_repo(self, repo_name: str) -> Repository:
+ """
+ :param repo_name: str
+ :return:
+ :rtype: Repository
+ """
+ try:
+ repo: Repository = self.gh.get_repo(repo_name)
+ except BadCredentialsException:
+ print(f'Credentials from {ENV_GITHUB_TOKEN} were bad.')
+ sys.exit(1)
+ return repo
+
+ @staticmethod
+ def get_major_version(from_go_version: str) -> str:
+ """
+ :param from_go_version: str
+ :return:
+ :rtype: str
+ """
+ return re.search(pattern=r'^\d+\.\d+', string=from_go_version).group(0)
+
+ def get_latest_major_upgrade(self, from_go_version: str) -> str:
+ """
+ :param from_go_version: str
+ :return:
+ :rtype: str
+ """
+ major_version = self.get_major_version(from_go_version)
+ go_version_response: Response = requests.get(GO_VERSION_URL)
+ go_version_response.raise_for_status()
+ go_version_content: list = json.loads(go_version_response.content)
+ index = 0
+ fetched_go_version: str = ''
+ while True:
+ if not go_version_content[index]['stable']:
+ continue
+ go_version_name: str = go_version_content[index]['version']
+ fetched_go_version = re.search(pattern=r'[\d.]+', string=go_version_name).group(0)
+ if major_version == self.get_major_version(fetched_go_version):
+ break
+ index += 1
+ if major_version != self.get_major_version(fetched_go_version):
+ raise Exception(f'No supported {major_version} Go versions exist.')
+ print(f'Latest version of Go {major_version} is {fetched_go_version}')
+ return fetched_go_version
+
+ def get_repo_name(self) -> str:
+ """
+ :return:
+ :rtype: str
+ """
+ repo_name: str = self.getenv(ENV_GITHUB_REPOSITORY)
+ return repo_name
+
+ def get_repo_owner(self) -> str:
+ """
+ :return:
+ :rtype: str
+ """
+ repo_name: str = self.getenv(ENV_GITHUB_REPOSITORY_OWNER)
+ return repo_name
+
+ def get_go_milestone(self, go_version: str) -> str:
+ """
+ :param go_version: str
+ :return:
+ """
+ go_repo: Repository = self.get_repo(GO_REPO_NAME)
+ milestones: PaginatedList[Milestone] = go_repo.get_milestones(state='all', sort='due_on',
+ direction='desc')
+ milestone_title = f'Go{go_version}'
+ for milestone in milestones: # type: Milestone
+ if milestone.title == milestone_title:
+ print(f'Found Go milestone {milestone.title}')
+ return milestone.raw_data.get('html_url')
+ raise Exception(f'Could not find a milestone named {milestone_title}.')
+
+ @staticmethod
+ def get_release_notes_page() -> str:
+ """
+ :return:
+ :rtype: str
+ """
+ release_history_response: Response = requests.get(RELEASE_PAGE_URL)
+ release_history_response.raise_for_status()
+ return release_history_response.content.decode()
+
+ @staticmethod
+ def get_release_notes(go_version: str, release_notes_content: str) -> str:
+ """
+ :param go_version: str
+ :param release_notes_content: str
+ :return:
+ :rtype: str
+ """
+ go_version_pattern = go_version.replace('.', '\\.')
+ release_notes_pattern: str = f'<p>\\s*\\n\\s*go{go_version_pattern}.*?</p>'
+ release_notes_matches = re.search(release_notes_pattern, release_notes_content,
+ re.MULTILINE | re.DOTALL)
+ if release_notes_matches is None:
+ raise Exception(f'Could not find release notes on {RELEASE_PAGE_URL}')
+ release_notes = re.sub(r'[\s\t]+', ' ', release_notes_matches.group(0))
+ return release_notes
+
+ def get_pr_body(self, go_version: str, milestone_url: str) -> str:
+ """
+ :param go_version: str
+ :param milestone_url: str
+ :return:
+ :rtype: str
+ """
+ with open(os.path.dirname(__file__) + '/pr_template.md') as file:
+ pr_template = file.read()
+ go_major_version = self.get_major_version(go_version)
+
+ release_notes = self.get_release_notes(go_version, self.get_release_notes_page())
+ pr_body: str = pr_template.format(GO_VERSION=go_version, GO_MAJOR_VERSION=go_major_version,
+ RELEASE_NOTES=release_notes, MILESTONE_URL=milestone_url)
+ print('Templated PR body')
+ return pr_body
+
+ def get_repo_go_version(self, branch: str = 'master') -> str:
+ """
+ :param branch: str
+ :return:
+ :rtype: str
+ """
+ return self.repo.get_contents(self.getenv(ENV_GO_VERSION_FILE),
+ f'refs/heads/{branch}').decoded_content.rstrip().decode()
+
+ def set_go_version(self, go_version: str, commit_message: str,
+ source_branch_name: str) -> Commit:
+ """
+ :param go_version: str
+ :param commit_message: str
+ :param source_branch_name: str
+ :return:
+ :rtype: str
+ """
+ master: Branch = self.repo.get_branch('master')
+ sha: str = master.commit.sha
+ ref: str = f'refs/heads/{source_branch_name}'
+ self.repo.create_git_ref(ref, sha)
+
+ print(f'Created branch {source_branch_name}')
+ go_version_file: str = self.getenv(ENV_GO_VERSION_FILE)
+ go_file_contents = self.repo.get_contents(go_version_file, ref)
+ kwargs = {'path': go_version_file,
+ 'message': commit_message,
+ 'content': (go_version + '\n'),
+ 'sha': go_file_contents.sha,
+ 'branch': source_branch_name,
+ }
+ try:
+ git_author_name = self.getenv(ENV_GIT_AUTHOR_NAME)
+ git_author_email = GIT_AUTHOR_EMAIL_TEMPLATE.format(git_author_name=git_author_name)
+ author: InputGitAuthor = InputGitAuthor(name=git_author_name, email=git_author_email)
+ kwargs['author'] = author
+ kwargs['committer'] = author
+ except KeyError:
+ print('Committing using the default author')
+
+ commit: Commit = self.repo.update_file(**kwargs).get('commit')
+ print(f'Updated {go_version_file} on {self.repo.name}')
+ return commit
+
+ def update_golang_org_x(self, previous_commit: Commit) -> Union[GitCommit, None]:
+ """
+ :param previous_commit:
+ :type previous_commit:
+ :return:
+ :rtype: Union[GitCommit, None]
+ """
+ subprocess.run(['git', 'fetch', 'origin'], check=True)
+ subprocess.run(['git', 'checkout', previous_commit.sha], check=True)
+ script_path: str = f'.github/actions/pr-to-update-go/update_golang_org_x.sh'
Review comment:
*f* removed in 978e5f94cd
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on pull request #5506: Workflow to check for and PR minor Go revision updates
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on pull request #5506:
URL: https://github.com/apache/trafficcontrol/pull/5506#issuecomment-777203220
Updated the workflow so that it additionally updates `golang.org/x/...` dependencies used by the project, since before Go modules, we always used the latest versions, and with Go modules, their versions are pinned.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #5506: Workflow to check for and PR minor Go revision updates
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on a change in pull request #5506:
URL: https://github.com/apache/trafficcontrol/pull/5506#discussion_r574052782
##########
File path: .github/actions/pr-to-update-go/pr_to_update_go/GoPRMaker.py
##########
@@ -0,0 +1,210 @@
+#!/usr/bin/env python3
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+import json
+import os
+import re
+import requests
+import sys
+
+from github.InputGitAuthor import InputGitAuthor
+from github.Label import Label
+from requests import Response
+
+from github.Branch import Branch
+from github.GithubException import BadCredentialsException, GithubException, UnknownObjectException
+from github.MainClass import Github
+from github.Milestone import Milestone
+from github.PaginatedList import PaginatedList
+from github.PullRequest import PullRequest
+from github.Repository import Repository
+
+from pr_to_update_go.constants import ENV_GITHUB_TOKEN, GO_VERSION_URL, ENV_GITHUB_REPOSITORY, \
+ ENV_GITHUB_REPOSITORY_OWNER, GO_REPO_NAME, RELEASE_PAGE_URL, ENV_GO_VERSION_FILE, ENV_GIT_AUTHOR_NAME, \
+ GIT_AUTHOR_EMAIL_TEMPLATE
+
+
+class GoPRMaker:
+ gh: Github
+ latest_go_version: str
+ repo: Repository
+
+ def __init__(self, gh: Github) -> None:
+ self.gh = gh
+ repo_name: str = self.get_repo_name()
+ self.repo = self.get_repo(repo_name)
+
+ def run(self) -> None:
Review comment:
I like `-> None:` because then PyCharm gives you warnings when trying to use a value returned by a function whose return type is `None` without needing to analyze control flow. For example, on line 6 of this snippet,
```python
def a() -> None:
if False:
return 1
print('Returns nothing')
b: int = a()
```
it warns
> Expected type 'int', got 'None' instead.
But that warning is gone if `-> None` is removed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #5506: Workflow to check for and PR minor Go revision updates
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #5506:
URL: https://github.com/apache/trafficcontrol/pull/5506#discussion_r574124600
##########
File path: .github/actions/pr-to-update-go/pr_to_update_go/GoPRMaker.py
##########
@@ -0,0 +1,210 @@
+#!/usr/bin/env python3
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+import json
+import os
+import re
+import requests
+import sys
+
+from github.InputGitAuthor import InputGitAuthor
+from github.Label import Label
+from requests import Response
+
+from github.Branch import Branch
+from github.GithubException import BadCredentialsException, GithubException, UnknownObjectException
+from github.MainClass import Github
+from github.Milestone import Milestone
+from github.PaginatedList import PaginatedList
+from github.PullRequest import PullRequest
+from github.Repository import Repository
+
+from pr_to_update_go.constants import ENV_GITHUB_TOKEN, GO_VERSION_URL, ENV_GITHUB_REPOSITORY, \
+ ENV_GITHUB_REPOSITORY_OWNER, GO_REPO_NAME, RELEASE_PAGE_URL, ENV_GO_VERSION_FILE, ENV_GIT_AUTHOR_NAME, \
+ GIT_AUTHOR_EMAIL_TEMPLATE
+
+
+class GoPRMaker:
+ gh: Github
+ latest_go_version: str
+ repo: Repository
+
+ def __init__(self, gh: Github) -> None:
+ self.gh = gh
+ repo_name: str = self.get_repo_name()
+ self.repo = self.get_repo(repo_name)
+
+ def run(self) -> None:
Review comment:
So Pycharm treats implicit return types as [Any](https://docs.python.org/3/library/typing.html#typing.Any) ? As a Typescript programmer I turn my nose up at such sloppy inference!
But that's fine.
It's weird to me that `typing` has an `any` and a `never` but not a `void`. Ideally you'd be able to do
```python3
from typing import Any, Void
def a() -> Void:
pass
b: Any = a()
```
and that would show up in linters/IDEs/whathaveyou as an invalid assignment of void to a value. But I guess the technology just isn't there yet.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #5506: Workflow to check for and PR minor Go revision updates
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #5506:
URL: https://github.com/apache/trafficcontrol/pull/5506#discussion_r573942106
##########
File path: .github/actions/pr-to-update-go/README.rst
##########
@@ -0,0 +1,73 @@
+..
+..
+.. Licensed under the Apache License, Version 2.0 (the "License");
+.. you may not use this file except in compliance with the License.
+.. You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+..
+
+***************
+pr-to-update-go
+***************
+
+Opens a PR if a new minor Go revision is available.
+
+For example, it will open a PR if the ``GO_VERSION`` contains ``1.14.7`` but Go versions 1.15.1 and 1.14.8 are available, it will
+
+1. Create a branch named ``go-1.14.8`` to update the repo's Go version to 1.14.8
+2. Open a PR targeting the ``master`` branch from branch ``go-1.14.8``
+
+Other behavior in this scenario:
+
+- If a branch named ``go-1.14.8`` already exists, no additional branch is created.
+- If a PR titled *Update Go version to 1.14.8* already exists, no additional PR is opened.
+
+Environment Variables
+=====================
+
++----------------------------+----------------------------------------------------------------------------------+
+| Environment Variable Name | Value |
++============================+==================================================================================+
+| ``GIT_AUTHOR_NAME`` | Optional. The username to associate with the commit that updates the Go version. |
++----------------------------+----------------------------------------------------------------------------------+
+| ``GITHUB_TOKEN`` | Required. ``${{ github.token }}`` or ``${{ secrets.GITHUB_TOKEN }}`` |
++----------------------------+----------------------------------------------------------------------------------+
+| ``GO_VERSION_FILE`` | Required. The file in the repo containing the version of Go used by the repo. |
++----------------------------+----------------------------------------------------------------------------------+
+
+
+Outputs
+=======
+
+``exit-code``
+-------------
+
+Exit code is 0 unless an error was encountered.
+
+Example usage
+=============
+
+.. code-block:: yaml
+
+ - name: PR to Update Go
+ run: python3 -m pr_to_update_go
+ env:
+ GIT_AUTHOR_NAME: asfgit
+ GITHUB_TOKEN: ${{ github.token }}
+ GO_VERSION_FILE: GO_VERSION
+
+Tests
+=====
+
+To run the unit tests:
+
+.. code-block:: shell
+
+ python3 -m unittest discover ./tests
Review comment:
If the intended parser is PyPi it also won't work (I know from my experience packaging the TO Python client), but I think that's probably less likely.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #5506: Workflow to check for and PR minor Go revision updates
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #5506:
URL: https://github.com/apache/trafficcontrol/pull/5506#discussion_r574121282
##########
File path: .github/actions/pr-to-update-go/README.rst
##########
@@ -0,0 +1,73 @@
+..
+..
+.. Licensed under the Apache License, Version 2.0 (the "License");
+.. you may not use this file except in compliance with the License.
+.. You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+..
+
+***************
+pr-to-update-go
+***************
+
+Opens a PR if a new minor Go revision is available.
+
+For example, it will open a PR if the ``GO_VERSION`` contains ``1.14.7`` but Go versions 1.15.1 and 1.14.8 are available, it will
+
+1. Create a branch named ``go-1.14.8`` to update the repo's Go version to 1.14.8
+2. Open a PR targeting the ``master`` branch from branch ``go-1.14.8``
+
+Other behavior in this scenario:
+
+- If a branch named ``go-1.14.8`` already exists, no additional branch is created.
+- If a PR titled *Update Go version to 1.14.8* already exists, no additional PR is opened.
+
+Environment Variables
+=====================
+
++----------------------------+----------------------------------------------------------------------------------+
+| Environment Variable Name | Value |
++============================+==================================================================================+
+| ``GIT_AUTHOR_NAME`` | Optional. The username to associate with the commit that updates the Go version. |
++----------------------------+----------------------------------------------------------------------------------+
+| ``GITHUB_TOKEN`` | Required. ``${{ github.token }}`` or ``${{ secrets.GITHUB_TOKEN }}`` |
++----------------------------+----------------------------------------------------------------------------------+
+| ``GO_VERSION_FILE`` | Required. The file in the repo containing the version of Go used by the repo. |
++----------------------------+----------------------------------------------------------------------------------+
+
+
+Outputs
+=======
+
+``exit-code``
+-------------
+
+Exit code is 0 unless an error was encountered.
+
+Example usage
+=============
+
+.. code-block:: yaml
+
+ - name: PR to Update Go
+ run: python3 -m pr_to_update_go
+ env:
+ GIT_AUTHOR_NAME: asfgit
+ GITHUB_TOKEN: ${{ github.token }}
+ GO_VERSION_FILE: GO_VERSION
+
+Tests
+=====
+
+To run the unit tests:
+
+.. code-block:: shell
+
+ python3 -m unittest discover ./tests
Review comment:
That's very interesting, because if you look at e.g. [the source for the `/cdns` API endpoint docs in APIv4](https://github.com/apache/trafficcontrol/blob/b3a6c32500f0c69eb81b480cba9d98da91dfa568/docs/source/api/v4/cdns.rst) you'll notice it isn't rendering request or response examples.
Matter of syntax support? You'd think it just falls back to regular monospace...
Regardless, you're right, it's rendering properly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 merged pull request #5506: Workflow to check for and PR minor Go revision updates
Posted by GitBox <gi...@apache.org>.
ocket8888 merged pull request #5506:
URL: https://github.com/apache/trafficcontrol/pull/5506
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #5506: Workflow to check for and PR minor Go revision updates
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on a change in pull request #5506:
URL: https://github.com/apache/trafficcontrol/pull/5506#discussion_r574053069
##########
File path: .github/actions/pr-to-update-go/README.rst
##########
@@ -0,0 +1,73 @@
+..
+..
+.. Licensed under the Apache License, Version 2.0 (the "License");
+.. you may not use this file except in compliance with the License.
+.. You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+..
+
+***************
+pr-to-update-go
+***************
+
+Opens a PR if a new minor Go revision is available.
+
+For example, it will open a PR if the ``GO_VERSION`` contains ``1.14.7`` but Go versions 1.15.1 and 1.14.8 are available, it will
+
+1. Create a branch named ``go-1.14.8`` to update the repo's Go version to 1.14.8
+2. Open a PR targeting the ``master`` branch from branch ``go-1.14.8``
Review comment:
Sentence restructured in 366a14ebf
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] zrhoffman commented on a change in pull request #5506: Workflow to check for and PR minor Go revision updates
Posted by GitBox <gi...@apache.org>.
zrhoffman commented on a change in pull request #5506:
URL: https://github.com/apache/trafficcontrol/pull/5506#discussion_r574052926
##########
File path: .github/actions/pr-to-update-go/README.rst
##########
@@ -0,0 +1,73 @@
+..
+..
+.. Licensed under the Apache License, Version 2.0 (the "License");
+.. you may not use this file except in compliance with the License.
+.. You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+..
+
+***************
+pr-to-update-go
+***************
+
+Opens a PR if a new minor Go revision is available.
+
+For example, it will open a PR if the ``GO_VERSION`` contains ``1.14.7`` but Go versions 1.15.1 and 1.14.8 are available, it will
+
+1. Create a branch named ``go-1.14.8`` to update the repo's Go version to 1.14.8
+2. Open a PR targeting the ``master`` branch from branch ``go-1.14.8``
+
+Other behavior in this scenario:
+
+- If a branch named ``go-1.14.8`` already exists, no additional branch is created.
+- If a PR titled *Update Go version to 1.14.8* already exists, no additional PR is opened.
+
+Environment Variables
+=====================
+
++----------------------------+----------------------------------------------------------------------------------+
+| Environment Variable Name | Value |
++============================+==================================================================================+
+| ``GIT_AUTHOR_NAME`` | Optional. The username to associate with the commit that updates the Go version. |
++----------------------------+----------------------------------------------------------------------------------+
+| ``GITHUB_TOKEN`` | Required. ``${{ github.token }}`` or ``${{ secrets.GITHUB_TOKEN }}`` |
++----------------------------+----------------------------------------------------------------------------------+
+| ``GO_VERSION_FILE`` | Required. The file in the repo containing the version of Go used by the repo. |
++----------------------------+----------------------------------------------------------------------------------+
+
+
+Outputs
+=======
+
+``exit-code``
+-------------
+
+Exit code is 0 unless an error was encountered.
+
+Example usage
+=============
+
+.. code-block:: yaml
+
+ - name: PR to Update Go
+ run: python3 -m pr_to_update_go
+ env:
+ GIT_AUTHOR_NAME: asfgit
+ GITHUB_TOKEN: ${{ github.token }}
+ GO_VERSION_FILE: GO_VERSION
+
+Tests
+=====
+
+To run the unit tests:
+
+.. code-block:: shell
+
+ python3 -m unittest discover ./tests
Review comment:
> However, GitHub's renderer isn't Sphinx. I think it's just standard RST - and the `code-block` directive is provided by Sphinx, so if the intention was for this to be readable on GitHub I don't think it'll work.
I wasn't able to find anything rendering incorrectly at https://github.com/apache/trafficcontrol/blob/b3a6c32500/.github/actions/pr-to-update-go/README.rst.
Sphinx doesn't seem to care about about a blank line after
https://github.com/apache/trafficcontrol/blob/b3a6c32500f0c69eb81b480cba9d98da91dfa568/.github/actions/pr-to-update-go/README.rst#L73
When using Sphinx, the code block renders correctly with or without a newline at the end of the file.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #5506: Workflow to check for and PR minor Go revision updates
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #5506:
URL: https://github.com/apache/trafficcontrol/pull/5506#discussion_r577054649
##########
File path: .github/actions/pr-to-update-go/pr_to_update_go/go_pr_maker.py
##########
@@ -0,0 +1,391 @@
+#!/usr/bin/env python3
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+"""
+Generate pull requests that update a repository's Go version.
+
+Classes:
+
+ GoPRMaker
+
+"""
+import json
+import os
+import re
+import subprocess
+import sys
+from typing import Union
+
+import requests
+from github.Branch import Branch
+from github.Commit import Commit
+from github.GitCommit import GitCommit
+from github.GitRef import GitRef
Review comment:
This import is never used
##########
File path: .github/actions/pr-to-update-go/pr_to_update_go/go_pr_maker.py
##########
@@ -0,0 +1,391 @@
+#!/usr/bin/env python3
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+"""
+Generate pull requests that update a repository's Go version.
+
+Classes:
+
+ GoPRMaker
+
+"""
+import json
+import os
+import re
+import subprocess
+import sys
+from typing import Union
+
+import requests
+from github.Branch import Branch
+from github.Commit import Commit
+from github.GitCommit import GitCommit
+from github.GitRef import GitRef
+from github.GitTree import GitTree
+from github.GithubObject import NotSet
+from github.InputGitTreeElement import InputGitTreeElement
+from github.Requester import Requester
+
+from requests import Response
+
+from github.GithubException import BadCredentialsException, GithubException, UnknownObjectException
+from github.InputGitAuthor import InputGitAuthor
+from github.Label import Label
+from github.MainClass import Github
+from github.Milestone import Milestone
+from github.PaginatedList import PaginatedList
+from github.PullRequest import PullRequest
+from github.Repository import Repository
+
+from pr_to_update_go.constants import ENV_GITHUB_TOKEN, GO_VERSION_URL, ENV_GITHUB_REPOSITORY, \
+ ENV_GITHUB_REPOSITORY_OWNER, GO_REPO_NAME, RELEASE_PAGE_URL, ENV_GO_VERSION_FILE, \
+ ENV_GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL_TEMPLATE
+
+
+class GoPRMaker:
+ """
+ A class to generate pull requests for the purpose of updating the Go version in a repository.
+ """
+ gh: Github
+ latest_go_version: str
+ repo: Repository
+ author: InputGitAuthor
+
+ def __init__(self, gh: Github) -> None:
+ """
+ :param gh: Github
+ :rtype: None
+ """
+ self.gh = gh
+ repo_name: str = self.get_repo_name()
+ self.repo = self.get_repo(repo_name)
+
+ try:
+ git_author_name = self.getenv(ENV_GIT_AUTHOR_NAME)
+ git_author_email = GIT_AUTHOR_EMAIL_TEMPLATE.format(git_author_name=git_author_name)
+ self.author = InputGitAuthor(git_author_name, git_author_email)
+ except KeyError:
+ self.author = NotSet
+ print('Will commit using the default author')
+
+ def branch_exists(self, branch: str) -> bool:
+ """
+ :param branch:
+ :type branch:
+ :return:
+ :rtype: bool
+ """
+ try:
+ repo_go_version = self.get_repo_go_version(branch)
+ if self.latest_go_version == repo_go_version:
+ print(f'Branch {branch} already exists')
+ return True
+ except GithubException as e:
+ message = e.data.get('message')
+ if not re.match(r'No commit found for the ref', message):
+ raise e
+ return False
+
+ def update_branch(self, branch_name: str, sha: str) -> None:
+ """
+ :param branch_name:
+ :type branch_name:
+ :param sha:
+ :type sha:
+ :return:
+ :rtype: None
+ """
+ requester: Requester = self.repo._requester
+ patch_parameters = {
+ 'sha': sha,
+ }
+ requester.requestJsonAndCheck(
+ 'PATCH', self.repo.url + f'/git/refs/heads/{branch_name}', input=patch_parameters
+ )
+ return
+
+ def run(self) -> None:
+ """
+ :return:
+ :rtype: None
+ """
+ repo_go_version = self.get_repo_go_version()
+ self.latest_go_version = self.get_latest_major_upgrade(repo_go_version)
+ commit_message: str = f'Update Go version to {self.latest_go_version}'
+
+ source_branch_name: str = f'go-{self.latest_go_version}'
+ target_branch: str = 'master'
+ if repo_go_version == self.latest_go_version:
+ print(f'Go version is up-to-date on {target_branch}, nothing to do.')
+ return
+
+ if not self.branch_exists(source_branch_name):
+ commit: Commit = self.set_go_version(self.latest_go_version, commit_message,
+ source_branch_name)
+ update_golang_org_x_commit: Union[GitCommit, None] = self.update_golang_org_x(commit)
+ if isinstance(update_golang_org_x_commit, GitCommit):
+ sha: str = update_golang_org_x_commit.sha
+ self.update_branch(source_branch_name, sha)
+
+ owner: str = self.get_repo_owner()
+ self.create_pr(self.latest_go_version, commit_message, owner, source_branch_name,
+ target_branch)
+
+ @staticmethod
+ def getenv(env_name: str) -> str:
+ """
+ :param env_name: str
+ :return:
+ :rtype: str
+ """
+ return os.environ[env_name]
+
+ def get_repo(self, repo_name: str) -> Repository:
+ """
+ :param repo_name: str
+ :return:
+ :rtype: Repository
+ """
+ try:
+ repo: Repository = self.gh.get_repo(repo_name)
+ except BadCredentialsException:
+ print(f'Credentials from {ENV_GITHUB_TOKEN} were bad.')
+ sys.exit(1)
+ return repo
+
+ @staticmethod
+ def get_major_version(from_go_version: str) -> str:
+ """
+ :param from_go_version: str
+ :return:
+ :rtype: str
+ """
+ return re.search(pattern=r'^\d+\.\d+', string=from_go_version).group(0)
+
+ def get_latest_major_upgrade(self, from_go_version: str) -> str:
+ """
+ :param from_go_version: str
+ :return:
+ :rtype: str
+ """
+ major_version = self.get_major_version(from_go_version)
+ go_version_response: Response = requests.get(GO_VERSION_URL)
+ go_version_response.raise_for_status()
+ go_version_content: list = json.loads(go_version_response.content)
+ index = 0
+ fetched_go_version: str = ''
+ while True:
+ if not go_version_content[index]['stable']:
+ continue
+ go_version_name: str = go_version_content[index]['version']
+ fetched_go_version = re.search(pattern=r'[\d.]+', string=go_version_name).group(0)
+ if major_version == self.get_major_version(fetched_go_version):
+ break
+ index += 1
+ if major_version != self.get_major_version(fetched_go_version):
+ raise Exception(f'No supported {major_version} Go versions exist.')
+ print(f'Latest version of Go {major_version} is {fetched_go_version}')
+ return fetched_go_version
+
+ def get_repo_name(self) -> str:
+ """
+ :return:
+ :rtype: str
+ """
+ repo_name: str = self.getenv(ENV_GITHUB_REPOSITORY)
+ return repo_name
+
+ def get_repo_owner(self) -> str:
+ """
+ :return:
+ :rtype: str
+ """
+ repo_name: str = self.getenv(ENV_GITHUB_REPOSITORY_OWNER)
+ return repo_name
+
+ def get_go_milestone(self, go_version: str) -> str:
+ """
+ :param go_version: str
+ :return:
+ """
+ go_repo: Repository = self.get_repo(GO_REPO_NAME)
+ milestones: PaginatedList[Milestone] = go_repo.get_milestones(state='all', sort='due_on',
+ direction='desc')
+ milestone_title = f'Go{go_version}'
+ for milestone in milestones: # type: Milestone
+ if milestone.title == milestone_title:
+ print(f'Found Go milestone {milestone.title}')
+ return milestone.raw_data.get('html_url')
+ raise Exception(f'Could not find a milestone named {milestone_title}.')
+
+ @staticmethod
+ def get_release_notes_page() -> str:
+ """
+ :return:
+ :rtype: str
+ """
+ release_history_response: Response = requests.get(RELEASE_PAGE_URL)
+ release_history_response.raise_for_status()
+ return release_history_response.content.decode()
+
+ @staticmethod
+ def get_release_notes(go_version: str, release_notes_content: str) -> str:
+ """
+ :param go_version: str
+ :param release_notes_content: str
+ :return:
+ :rtype: str
+ """
+ go_version_pattern = go_version.replace('.', '\\.')
+ release_notes_pattern: str = f'<p>\\s*\\n\\s*go{go_version_pattern}.*?</p>'
+ release_notes_matches = re.search(release_notes_pattern, release_notes_content,
+ re.MULTILINE | re.DOTALL)
+ if release_notes_matches is None:
+ raise Exception(f'Could not find release notes on {RELEASE_PAGE_URL}')
+ release_notes = re.sub(r'[\s\t]+', ' ', release_notes_matches.group(0))
+ return release_notes
+
+ def get_pr_body(self, go_version: str, milestone_url: str) -> str:
+ """
+ :param go_version: str
+ :param milestone_url: str
+ :return:
+ :rtype: str
+ """
+ with open(os.path.dirname(__file__) + '/pr_template.md') as file:
+ pr_template = file.read()
+ go_major_version = self.get_major_version(go_version)
+
+ release_notes = self.get_release_notes(go_version, self.get_release_notes_page())
+ pr_body: str = pr_template.format(GO_VERSION=go_version, GO_MAJOR_VERSION=go_major_version,
+ RELEASE_NOTES=release_notes, MILESTONE_URL=milestone_url)
+ print('Templated PR body')
+ return pr_body
+
+ def get_repo_go_version(self, branch: str = 'master') -> str:
+ """
+ :param branch: str
+ :return:
+ :rtype: str
+ """
+ return self.repo.get_contents(self.getenv(ENV_GO_VERSION_FILE),
+ f'refs/heads/{branch}').decoded_content.rstrip().decode()
+
+ def set_go_version(self, go_version: str, commit_message: str,
+ source_branch_name: str) -> Commit:
+ """
+ :param go_version: str
+ :param commit_message: str
+ :param source_branch_name: str
+ :return:
+ :rtype: str
+ """
+ master: Branch = self.repo.get_branch('master')
+ sha: str = master.commit.sha
+ ref: str = f'refs/heads/{source_branch_name}'
+ self.repo.create_git_ref(ref, sha)
+
+ print(f'Created branch {source_branch_name}')
+ go_version_file: str = self.getenv(ENV_GO_VERSION_FILE)
+ go_file_contents = self.repo.get_contents(go_version_file, ref)
+ kwargs = {'path': go_version_file,
+ 'message': commit_message,
+ 'content': (go_version + '\n'),
+ 'sha': go_file_contents.sha,
+ 'branch': source_branch_name,
+ }
+ try:
+ git_author_name = self.getenv(ENV_GIT_AUTHOR_NAME)
+ git_author_email = GIT_AUTHOR_EMAIL_TEMPLATE.format(git_author_name=git_author_name)
+ author: InputGitAuthor = InputGitAuthor(name=git_author_name, email=git_author_email)
+ kwargs['author'] = author
+ kwargs['committer'] = author
+ except KeyError:
+ print('Committing using the default author')
+
+ commit: Commit = self.repo.update_file(**kwargs).get('commit')
+ print(f'Updated {go_version_file} on {self.repo.name}')
+ return commit
+
+ def update_golang_org_x(self, previous_commit: Commit) -> Union[GitCommit, None]:
+ """
+ :param previous_commit:
+ :type previous_commit:
+ :return:
+ :rtype: Union[GitCommit, None]
+ """
+ subprocess.run(['git', 'fetch', 'origin'], check=True)
+ subprocess.run(['git', 'checkout', previous_commit.sha], check=True)
+ script_path: str = f'.github/actions/pr-to-update-go/update_golang_org_x.sh'
Review comment:
This f-string is unnecessary
##########
File path: .github/actions/pr-to-update-go/pr_to_update_go/go_pr_maker.py
##########
@@ -0,0 +1,391 @@
+#!/usr/bin/env python3
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+"""
+Generate pull requests that update a repository's Go version.
+
+Classes:
+
+ GoPRMaker
+
+"""
+import json
+import os
+import re
+import subprocess
+import sys
+from typing import Union
+
+import requests
+from github.Branch import Branch
+from github.Commit import Commit
+from github.GitCommit import GitCommit
+from github.GitRef import GitRef
+from github.GitTree import GitTree
+from github.GithubObject import NotSet
+from github.InputGitTreeElement import InputGitTreeElement
+from github.Requester import Requester
+
+from requests import Response
+
+from github.GithubException import BadCredentialsException, GithubException, UnknownObjectException
+from github.InputGitAuthor import InputGitAuthor
+from github.Label import Label
+from github.MainClass import Github
+from github.Milestone import Milestone
+from github.PaginatedList import PaginatedList
+from github.PullRequest import PullRequest
+from github.Repository import Repository
+
+from pr_to_update_go.constants import ENV_GITHUB_TOKEN, GO_VERSION_URL, ENV_GITHUB_REPOSITORY, \
+ ENV_GITHUB_REPOSITORY_OWNER, GO_REPO_NAME, RELEASE_PAGE_URL, ENV_GO_VERSION_FILE, \
+ ENV_GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL_TEMPLATE
+
+
+class GoPRMaker:
+ """
+ A class to generate pull requests for the purpose of updating the Go version in a repository.
+ """
+ gh: Github
+ latest_go_version: str
+ repo: Repository
+ author: InputGitAuthor
+
+ def __init__(self, gh: Github) -> None:
+ """
+ :param gh: Github
+ :rtype: None
+ """
+ self.gh = gh
+ repo_name: str = self.get_repo_name()
+ self.repo = self.get_repo(repo_name)
+
+ try:
+ git_author_name = self.getenv(ENV_GIT_AUTHOR_NAME)
+ git_author_email = GIT_AUTHOR_EMAIL_TEMPLATE.format(git_author_name=git_author_name)
+ self.author = InputGitAuthor(git_author_name, git_author_email)
+ except KeyError:
+ self.author = NotSet
+ print('Will commit using the default author')
+
+ def branch_exists(self, branch: str) -> bool:
+ """
+ :param branch:
+ :type branch:
+ :return:
+ :rtype: bool
+ """
+ try:
+ repo_go_version = self.get_repo_go_version(branch)
+ if self.latest_go_version == repo_go_version:
+ print(f'Branch {branch} already exists')
+ return True
+ except GithubException as e:
+ message = e.data.get('message')
+ if not re.match(r'No commit found for the ref', message):
+ raise e
+ return False
+
+ def update_branch(self, branch_name: str, sha: str) -> None:
+ """
+ :param branch_name:
+ :type branch_name:
+ :param sha:
+ :type sha:
+ :return:
+ :rtype: None
+ """
+ requester: Requester = self.repo._requester
+ patch_parameters = {
+ 'sha': sha,
+ }
+ requester.requestJsonAndCheck(
+ 'PATCH', self.repo.url + f'/git/refs/heads/{branch_name}', input=patch_parameters
+ )
+ return
+
+ def run(self) -> None:
+ """
+ :return:
+ :rtype: None
+ """
+ repo_go_version = self.get_repo_go_version()
+ self.latest_go_version = self.get_latest_major_upgrade(repo_go_version)
+ commit_message: str = f'Update Go version to {self.latest_go_version}'
+
+ source_branch_name: str = f'go-{self.latest_go_version}'
+ target_branch: str = 'master'
+ if repo_go_version == self.latest_go_version:
+ print(f'Go version is up-to-date on {target_branch}, nothing to do.')
+ return
+
+ if not self.branch_exists(source_branch_name):
+ commit: Commit = self.set_go_version(self.latest_go_version, commit_message,
+ source_branch_name)
+ update_golang_org_x_commit: Union[GitCommit, None] = self.update_golang_org_x(commit)
+ if isinstance(update_golang_org_x_commit, GitCommit):
+ sha: str = update_golang_org_x_commit.sha
+ self.update_branch(source_branch_name, sha)
+
+ owner: str = self.get_repo_owner()
+ self.create_pr(self.latest_go_version, commit_message, owner, source_branch_name,
+ target_branch)
+
+ @staticmethod
+ def getenv(env_name: str) -> str:
+ """
+ :param env_name: str
+ :return:
+ :rtype: str
+ """
+ return os.environ[env_name]
+
+ def get_repo(self, repo_name: str) -> Repository:
+ """
+ :param repo_name: str
+ :return:
+ :rtype: Repository
+ """
+ try:
+ repo: Repository = self.gh.get_repo(repo_name)
+ except BadCredentialsException:
+ print(f'Credentials from {ENV_GITHUB_TOKEN} were bad.')
+ sys.exit(1)
+ return repo
+
+ @staticmethod
+ def get_major_version(from_go_version: str) -> str:
+ """
+ :param from_go_version: str
+ :return:
+ :rtype: str
+ """
+ return re.search(pattern=r'^\d+\.\d+', string=from_go_version).group(0)
+
+ def get_latest_major_upgrade(self, from_go_version: str) -> str:
+ """
+ :param from_go_version: str
+ :return:
+ :rtype: str
+ """
+ major_version = self.get_major_version(from_go_version)
+ go_version_response: Response = requests.get(GO_VERSION_URL)
+ go_version_response.raise_for_status()
+ go_version_content: list = json.loads(go_version_response.content)
+ index = 0
+ fetched_go_version: str = ''
+ while True:
+ if not go_version_content[index]['stable']:
+ continue
+ go_version_name: str = go_version_content[index]['version']
+ fetched_go_version = re.search(pattern=r'[\d.]+', string=go_version_name).group(0)
+ if major_version == self.get_major_version(fetched_go_version):
+ break
+ index += 1
+ if major_version != self.get_major_version(fetched_go_version):
+ raise Exception(f'No supported {major_version} Go versions exist.')
+ print(f'Latest version of Go {major_version} is {fetched_go_version}')
+ return fetched_go_version
+
+ def get_repo_name(self) -> str:
+ """
+ :return:
+ :rtype: str
+ """
+ repo_name: str = self.getenv(ENV_GITHUB_REPOSITORY)
+ return repo_name
+
+ def get_repo_owner(self) -> str:
+ """
+ :return:
+ :rtype: str
+ """
+ repo_name: str = self.getenv(ENV_GITHUB_REPOSITORY_OWNER)
+ return repo_name
+
+ def get_go_milestone(self, go_version: str) -> str:
+ """
+ :param go_version: str
+ :return:
+ """
+ go_repo: Repository = self.get_repo(GO_REPO_NAME)
+ milestones: PaginatedList[Milestone] = go_repo.get_milestones(state='all', sort='due_on',
+ direction='desc')
+ milestone_title = f'Go{go_version}'
+ for milestone in milestones: # type: Milestone
+ if milestone.title == milestone_title:
+ print(f'Found Go milestone {milestone.title}')
+ return milestone.raw_data.get('html_url')
+ raise Exception(f'Could not find a milestone named {milestone_title}.')
+
+ @staticmethod
+ def get_release_notes_page() -> str:
+ """
+ :return:
+ :rtype: str
+ """
+ release_history_response: Response = requests.get(RELEASE_PAGE_URL)
+ release_history_response.raise_for_status()
+ return release_history_response.content.decode()
+
+ @staticmethod
+ def get_release_notes(go_version: str, release_notes_content: str) -> str:
+ """
+ :param go_version: str
+ :param release_notes_content: str
+ :return:
+ :rtype: str
+ """
+ go_version_pattern = go_version.replace('.', '\\.')
+ release_notes_pattern: str = f'<p>\\s*\\n\\s*go{go_version_pattern}.*?</p>'
+ release_notes_matches = re.search(release_notes_pattern, release_notes_content,
+ re.MULTILINE | re.DOTALL)
+ if release_notes_matches is None:
+ raise Exception(f'Could not find release notes on {RELEASE_PAGE_URL}')
+ release_notes = re.sub(r'[\s\t]+', ' ', release_notes_matches.group(0))
+ return release_notes
+
+ def get_pr_body(self, go_version: str, milestone_url: str) -> str:
+ """
+ :param go_version: str
+ :param milestone_url: str
+ :return:
+ :rtype: str
+ """
+ with open(os.path.dirname(__file__) + '/pr_template.md') as file:
+ pr_template = file.read()
+ go_major_version = self.get_major_version(go_version)
+
+ release_notes = self.get_release_notes(go_version, self.get_release_notes_page())
+ pr_body: str = pr_template.format(GO_VERSION=go_version, GO_MAJOR_VERSION=go_major_version,
+ RELEASE_NOTES=release_notes, MILESTONE_URL=milestone_url)
+ print('Templated PR body')
+ return pr_body
+
+ def get_repo_go_version(self, branch: str = 'master') -> str:
+ """
+ :param branch: str
+ :return:
+ :rtype: str
+ """
+ return self.repo.get_contents(self.getenv(ENV_GO_VERSION_FILE),
+ f'refs/heads/{branch}').decoded_content.rstrip().decode()
+
+ def set_go_version(self, go_version: str, commit_message: str,
+ source_branch_name: str) -> Commit:
+ """
+ :param go_version: str
+ :param commit_message: str
+ :param source_branch_name: str
+ :return:
+ :rtype: str
+ """
+ master: Branch = self.repo.get_branch('master')
+ sha: str = master.commit.sha
+ ref: str = f'refs/heads/{source_branch_name}'
+ self.repo.create_git_ref(ref, sha)
+
+ print(f'Created branch {source_branch_name}')
+ go_version_file: str = self.getenv(ENV_GO_VERSION_FILE)
+ go_file_contents = self.repo.get_contents(go_version_file, ref)
+ kwargs = {'path': go_version_file,
+ 'message': commit_message,
+ 'content': (go_version + '\n'),
+ 'sha': go_file_contents.sha,
+ 'branch': source_branch_name,
+ }
+ try:
+ git_author_name = self.getenv(ENV_GIT_AUTHOR_NAME)
+ git_author_email = GIT_AUTHOR_EMAIL_TEMPLATE.format(git_author_name=git_author_name)
+ author: InputGitAuthor = InputGitAuthor(name=git_author_name, email=git_author_email)
+ kwargs['author'] = author
+ kwargs['committer'] = author
+ except KeyError:
+ print('Committing using the default author')
+
+ commit: Commit = self.repo.update_file(**kwargs).get('commit')
+ print(f'Updated {go_version_file} on {self.repo.name}')
+ return commit
+
+ def update_golang_org_x(self, previous_commit: Commit) -> Union[GitCommit, None]:
+ """
+ :param previous_commit:
+ :type previous_commit:
+ :return:
+ :rtype: Union[GitCommit, None]
+ """
+ subprocess.run(['git', 'fetch', 'origin'], check=True)
+ subprocess.run(['git', 'checkout', previous_commit.sha], check=True)
+ script_path: str = f'.github/actions/pr-to-update-go/update_golang_org_x.sh'
+ subprocess.run([script_path], check=True)
+ files_to_check: list[str] = ['go.mod', 'go.sum', 'vendor/modules.txt']
+ tree_elements: list[InputGitTreeElement] = []
+ for file in files_to_check:
+ diff_process = subprocess.run(['git', 'diff', '--exit-code', '--', file])
+ if diff_process.returncode == 0:
+ continue
+ with open(file) as stream:
+ content: str = stream.read()
+ tree_element: InputGitTreeElement = InputGitTreeElement(path=file, mode='100644',
+ type='blob', content=content)
+ tree_elements.append(tree_element)
+ if len(tree_elements) == 0:
+ print('No golang.org/x/ dependencies need to be updated.')
+ return
+ tree_hash = subprocess.check_output(
+ ['git', 'log', '-1', '--pretty=%T', previous_commit.sha]).decode().strip()
+ base_tree: GitTree = self.repo.get_git_tree(sha=tree_hash)
+ tree: GitTree = self.repo.create_git_tree(tree_elements, base_tree)
+ commit_message: str = f'Update golang.org/x/ dependencies for go{self.latest_go_version}'
+ previous_git_commit: GitCommit = self.repo.get_git_commit(previous_commit.sha)
+ git_commit: GitCommit = self.repo.create_git_commit(message=commit_message, tree=tree,
+ parents=[previous_git_commit],
+ author=self.author, committer=self.author)
+ print(f'Updated golang.org/x/ dependencies')
Review comment:
This also doesn't need to be an f-string
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #5506: Workflow to check for and PR minor Go revision updates
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #5506:
URL: https://github.com/apache/trafficcontrol/pull/5506#discussion_r573398237
##########
File path: .github/actions/pr-to-update-go/README.rst
##########
@@ -0,0 +1,73 @@
+..
+..
+.. Licensed under the Apache License, Version 2.0 (the "License");
+.. you may not use this file except in compliance with the License.
+.. You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+..
+
+***************
+pr-to-update-go
+***************
+
+Opens a PR if a new minor Go revision is available.
+
+For example, it will open a PR if the ``GO_VERSION`` contains ``1.14.7`` but Go versions 1.15.1 and 1.14.8 are available, it will
+
+1. Create a branch named ``go-1.14.8`` to update the repo's Go version to 1.14.8
+2. Open a PR targeting the ``master`` branch from branch ``go-1.14.8``
Review comment:
Sentence structure here is a little repetitious: "it will open a PR if (condition), it will... [open a PR]"
##########
File path: .github/actions/pr-to-update-go/README.rst
##########
@@ -0,0 +1,73 @@
+..
+..
+.. Licensed under the Apache License, Version 2.0 (the "License");
+.. you may not use this file except in compliance with the License.
+.. You may obtain a copy of the License at
+..
+.. http://www.apache.org/licenses/LICENSE-2.0
+..
+.. Unless required by applicable law or agreed to in writing, software
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+..
+
+***************
+pr-to-update-go
+***************
+
+Opens a PR if a new minor Go revision is available.
+
+For example, it will open a PR if the ``GO_VERSION`` contains ``1.14.7`` but Go versions 1.15.1 and 1.14.8 are available, it will
+
+1. Create a branch named ``go-1.14.8`` to update the repo's Go version to 1.14.8
+2. Open a PR targeting the ``master`` branch from branch ``go-1.14.8``
+
+Other behavior in this scenario:
+
+- If a branch named ``go-1.14.8`` already exists, no additional branch is created.
+- If a PR titled *Update Go version to 1.14.8* already exists, no additional PR is opened.
+
+Environment Variables
+=====================
+
++----------------------------+----------------------------------------------------------------------------------+
+| Environment Variable Name | Value |
++============================+==================================================================================+
+| ``GIT_AUTHOR_NAME`` | Optional. The username to associate with the commit that updates the Go version. |
++----------------------------+----------------------------------------------------------------------------------+
+| ``GITHUB_TOKEN`` | Required. ``${{ github.token }}`` or ``${{ secrets.GITHUB_TOKEN }}`` |
++----------------------------+----------------------------------------------------------------------------------+
+| ``GO_VERSION_FILE`` | Required. The file in the repo containing the version of Go used by the repo. |
++----------------------------+----------------------------------------------------------------------------------+
+
+
+Outputs
+=======
+
+``exit-code``
+-------------
+
+Exit code is 0 unless an error was encountered.
+
+Example usage
+=============
+
+.. code-block:: yaml
+
+ - name: PR to Update Go
+ run: python3 -m pr_to_update_go
+ env:
+ GIT_AUTHOR_NAME: asfgit
+ GITHUB_TOKEN: ${{ github.token }}
+ GO_VERSION_FILE: GO_VERSION
+
+Tests
+=====
+
+To run the unit tests:
+
+.. code-block:: shell
+
+ python3 -m unittest discover ./tests
Review comment:
I believe you need a blank line at the end of a code block for Sphinx to parse it properly.
However, GitHub's renderer isn't Sphinx. I think it's just standard RST - and the `code-block` directive is provided by Sphinx, so if the intention was for this to be readable on GitHub I don't think it'll work.
##########
File path: .github/actions/pr-to-update-go/pr_to_update_go/__main__.py
##########
@@ -0,0 +1,32 @@
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+import os
+import sys
+
+from github.MainClass import Github
+
+from pr_to_update_go.GoPRMaker import GoPRMaker
+from pr_to_update_go.constants import ENV_GITHUB_TOKEN
+
+
+def main() -> None:
+ try:
+ github_token: str = os.environ[ENV_GITHUB_TOKEN]
+ except KeyError:
+ print(f'Environment variable {ENV_GITHUB_TOKEN} must be defined.')
+ sys.exit(1)
+ gh = Github(login_or_token=github_token)
+ GoPRMaker(gh).run()
+
+
+main()
Review comment:
I _think_ this should be wrapped in an `if __name__ == "__main__":` but I could be wrong; I'm not entirely sure the interpreter sets `__name__` like that when the module is called with `python -m`.
It's not normal to import modules beginning with `_` from packages - and nothing is meant to use this - so it could be safe, but it damages my delicate sensibilities to have large routines running as side effects to importing a module.
##########
File path: .github/actions/pr-to-update-go/pr_to_update_go/GoPRMaker.py
##########
@@ -0,0 +1,210 @@
+#!/usr/bin/env python3
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+import json
+import os
+import re
+import requests
+import sys
+
+from github.InputGitAuthor import InputGitAuthor
+from github.Label import Label
+from requests import Response
+
+from github.Branch import Branch
+from github.GithubException import BadCredentialsException, GithubException, UnknownObjectException
+from github.MainClass import Github
+from github.Milestone import Milestone
+from github.PaginatedList import PaginatedList
+from github.PullRequest import PullRequest
+from github.Repository import Repository
+
+from pr_to_update_go.constants import ENV_GITHUB_TOKEN, GO_VERSION_URL, ENV_GITHUB_REPOSITORY, \
+ ENV_GITHUB_REPOSITORY_OWNER, GO_REPO_NAME, RELEASE_PAGE_URL, ENV_GO_VERSION_FILE, ENV_GIT_AUTHOR_NAME, \
+ GIT_AUTHOR_EMAIL_TEMPLATE
+
+
+class GoPRMaker:
+ gh: Github
+ latest_go_version: str
+ repo: Repository
+
+ def __init__(self, gh: Github) -> None:
+ self.gh = gh
+ repo_name: str = self.get_repo_name()
+ self.repo = self.get_repo(repo_name)
+
+ def run(self) -> None:
Review comment:
This is just my two cents, but if a function doesn't return an explicit value it's fine to just leave it without a return type. I'm not sure our Pylint config cares, but to me a lack of return type is "void".
##########
File path: .github/actions/pr-to-update-go/pr_to_update_go/pr_template.md
##########
@@ -0,0 +1,120 @@
+<!--
+************ STOP!! ************
+If this Pull Request is intended to fix a security vulnerability, DO NOT submit it! Instead, contact
+the Apache Software Foundation Security Team at security@trafficcontrol.apache.org and follow the
+guidelines at https://www.apache.org/security/ regarding vulnerability disclosure.
+-->
+## What does this PR (Pull Request) do?
+<!-- Explain the changes you made here. If this fixes an Issue, identify it by
+replacing the text in the checkbox item with the Issue number e.g.
+
+- [x] This PR fixes #9001 OR is not related to any Issue
+
+^ This will automatically close Issue number 9001 when the Pull Request is
+merged (The '#' is important).
+
+Be sure you check the box properly, see the "The following criteria are ALL
+met by this PR" section for details.
+-->
Review comment:
I don't think you need comments like these, they're for humans to read
##########
File path: .github/actions/pr-to-update-go/setup.cfg
##########
@@ -0,0 +1,37 @@
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+[metadata]
+name = pr-to-update-go
+version = 0.0.0
+description = Opens a PR if a new minor Go revision is available.
+long_description = file: README.rst
+long_description_content_type = text/x-rst
+author = Apache Traffic Control
+author_email = dev@trafficcontrol.apache.org
+classifiers = OSI Approved :: Apache Software License
+
+[options]
+python_requires = >=3.9
+packages = pr_to_update_go
+install_requires =
+ PyGithub
+ requests
+
+[options.entry_points]
+console_scripts = pr-to-update-go = pr_to_update_go:main
+
+[options.extras_require]
+test = unittest
+
+[options.package_data]
+pr_to_update_go = pr_template.md
Review comment:
Setuptools will read this in lieu of passing the info as args? That's pretty neat. Wonder how long that's been the case.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficcontrol] ocket8888 commented on a change in pull request #5506: Workflow to check for and PR minor Go revision updates
Posted by GitBox <gi...@apache.org>.
ocket8888 commented on a change in pull request #5506:
URL: https://github.com/apache/trafficcontrol/pull/5506#discussion_r574125205
##########
File path: .github/actions/pr-to-update-go/pr_to_update_go/__main__.py
##########
@@ -0,0 +1,32 @@
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+import os
+import sys
+
+from github.MainClass import Github
+
+from pr_to_update_go.GoPRMaker import GoPRMaker
+from pr_to_update_go.constants import ENV_GITHUB_TOKEN
+
+
+def main() -> None:
+ try:
+ github_token: str = os.environ[ENV_GITHUB_TOKEN]
+ except KeyError:
+ print(f'Environment variable {ENV_GITHUB_TOKEN} must be defined.')
+ sys.exit(1)
+ gh = Github(login_or_token=github_token)
+ GoPRMaker(gh).run()
+
+
+main()
Review comment:
Oh, right. That's an interesting edge case.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org