You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/07/24 12:19:43 UTC
[GitHub] [ozone] adoroszlai opened a new pull request, #3620: HDDS-7044. Ignore pr_title_check for selective checks
adoroszlai opened a new pull request, #3620:
URL: https://github.com/apache/ozone/pull/3620
## What changes were proposed in this pull request?
Avoid triggering all tests:
* for workflows except `post-commit.yml`: they are not exercised by any jobs in the `post-commit.yml` (`build-branch`) workflow, so running those jobs is useless
* for `pr_title_check.*`: these are covered by specific checks (_rat_ for license, and _bats_ for bash unit tests)
https://issues.apache.org/jira/browse/HDDS-7044
## How was this patch tested?
Added test case in `selective_ci_checks.bats`. Also checked output for the commit for HDDS-7043 locally:
```
$ bash dev-support/ci/selective_ci_checks.sh 96009ab20
...
All 2 changed files are known to be handled by specific checks.
needs-basic-checks=true
basic-checks=["rat","bats"]
needs-build=false
needs-compile=false
needs-compose-tests=false
needs-dependency-check=false
needs-integration-tests=false
needs-kubernetes-tests=false
```
and for a commit that only changed the misc. workflows:
```
$ bash dev-support/ci/selective_ci_checks.sh 1493541d7
...
All 2 changed files are known to be handled by specific checks.
needs-basic-checks=true
basic-checks=["rat"]
needs-build=false
needs-compile=false
needs-compose-tests=false
needs-dependency-check=false
needs-integration-tests=false
needs-kubernetes-tests=false
```
Regular CI:
https://github.com/adoroszlai/hadoop-ozone/runs/7486998639
https://github.com/adoroszlai/hadoop-ozone/runs/7487000326
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #3620: HDDS-7044. Ignore pr_title_check for selective checks
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on PR #3620:
URL: https://github.com/apache/ozone/pull/3620#issuecomment-1198943210
@kerneltime @smengcl Thanks for taking a look. Let me try to give some background for the change.
1. We have different Github Actions workflows under `.github/workflows`:
* regular build/test CI is defined in `post-commit.yml`, triggered by pushes, PRs and on schedule
* PR titles are checked by `pull-request.yml`
* special PR comments ([example](https://github.com/apache/ozone/pull/3058#issuecomment-1193734431)) are handled by `comments.yaml`
* pending PRs are closed by `close-pending.yaml` on schedule
2. Regular CI (`post-commit.yml`) has some logic to decide whether to execute or skip specific checks (e.g. checkstyle, unit, acceptance, etc.) depending on which files were changed. This is implemented in `selective_ci_checks.sh`, which is run as the first job. It defines include and exclude regex patterns for each kind of check. These patterns are matched against the list of changed files. The script outputs true/false for each check, which is then picked up by the rest of the `post-commit.yml` workflow to skip/run other jobs.
3. There is also a "must run all checks" case, triggered by either of two conditions:
* if the change affects CI files, e.g. the workflow definition itself
* if some changed files are not handled by any of the check-specific patterns
4. We also run all checks for pushes and schedule, in other words, selective checks only applies to pull requests. (Example: compare workflow runs for the same [change](https://github.com/dombizita/ozone/commit/e6d9c06e6ca40c0b2b95f091a56299440d3426b4) when [pushed to developer's fork](https://github.com/dombizita/ozone/actions/runs/2669593534), [pull request is created](https://github.com/apache/ozone/actions/runs/2669631613), and finally [merged to `master`](https://github.com/apache/ozone/actions/runs/2671240589).)
Consider the case that I would like to change one of the other (non-`post-commit.yml`) workflow files, e.g. `pull-request.yml`. When I open a PR, this kind of change currently triggers all checks in the `post-commit.yml` CI workflow. Yet, most of the checks in `post-commit.yml` have nothing to do with `pull-request.yml`. None of the unit, integration or acceptance tests are affected. Checkstyle and findbugs (Java-specific checks) are not applicable.
So this PR tweaks some of the include/exclude patterns to be more specific.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #3620: HDDS-7044. Ignore pr_title_check for selective checks
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on PR #3620:
URL: https://github.com/apache/ozone/pull/3620#issuecomment-1203726408
Thanks @smengcl for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai merged pull request #3620: HDDS-7044. Ignore pr_title_check for selective checks
Posted by GitBox <gi...@apache.org>.
adoroszlai merged PR #3620:
URL: https://github.com/apache/ozone/pull/3620
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] smengcl commented on a diff in pull request #3620: HDDS-7044. Ignore pr_title_check for selective checks
Posted by GitBox <gi...@apache.org>.
smengcl commented on code in PR #3620:
URL: https://github.com/apache/ozone/pull/3620#discussion_r931518421
##########
dev-support/ci/selective_ci_checks.sh:
##########
@@ -190,10 +192,13 @@ function check_if_tests_are_needed_at_all() {
function run_all_tests_if_environment_files_changed() {
start_end::group_start "Check if everything should be run"
local pattern_array=(
- "^.github/workflows/"
+ "^.github/workflows/post-commit.yml"
Review Comment:
Does this mean we are only including the `post-commit.yml` rather than all the files under `.github/workflows/` with the change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kerneltime commented on pull request #3620: HDDS-7044. Ignore pr_title_check for selective checks
Posted by GitBox <gi...@apache.org>.
kerneltime commented on PR #3620:
URL: https://github.com/apache/ozone/pull/3620#issuecomment-1197267294
@adoroszlai I will need to sync up with you to understand this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org