You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudberry.apache.org by "edespino (via GitHub)" <gi...@apache.org> on 2024/11/27 10:19:18 UTC
[PR] Adding additional test suites. [cloudberry]
edespino opened a new pull request, #735:
URL: https://github.com/apache/cloudberry/pull/735
FYI: I created another PR so the tests could run. I didn't realize that the included .asf.yaml dynamically created new checks that are in this PR.
Enabled the following suites:
- `ic-singlenode_isolation`
- `ic-singlenode_isolation2`
- `ic-resgroup-v2`
Add step to display the contents of regression.diffs (if it exists)
Refactor check-skip job to handle PR-specific skip logic
- Updated GitHub Actions check-skip job:
- Modified logic to ensure skip behavior applies only to pull requests.
- Enhanced message parsing to include both PR titles and bodies.
- Ignored skip flags in direct pushes to main, ensuring full builds.
- Incorporated proper escaping of input data to prevent command injection.
This update improves the reliability and security of the check-skip workflow, aligning it with project requirements to trigger full builds for direct pushes while respecting skip flags in PRs.
Enhance GitHub Actions workflow with ignored test support
- Updated `build-cloudberry.yml`:
- Added parsing and reporting of ignored tests in the `test` job.
- Displayed names of ignored tests in test summaries.
- Included regression diffs display to aid debugging of test failures.
- Ensured consistent handling of failed and ignored tests across jobs.
- Updated workflow header:
- Documented ignored test handling in the test matrix.
- Highlighted regression diffs display as a debugging feature.
- Clarified reporting of failed and ignored tests in the workflow overview.
These changes improve the visibility and debugging of test results, ensuring ignored tests are appropriately tracked and reported without impacting the build status.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Adding additional test suites. [cloudberry]
Posted by "edespino (via GitHub)" <gi...@apache.org>.
edespino commented on code in PR #735:
URL: https://github.com/apache/cloudberry/pull/735#discussion_r1860424432
##########
.github/workflows/build-cloudberry.yml:
##########
@@ -17,102 +17,86 @@
# permissions and limitations under the License.
#
# --------------------------------------------------------------------
+# GitHub Actions Workflow: Apache Cloudberry Build Pipeline
+# --------------------------------------------------------------------
+# Description:
#
-# GitHub Actions Workflow: Apache Cloudberry Build
-# Description: Builds, tests, and packages Apache Cloudberry on Rocky Linux 9
+# This workflow builds, tests, and packages Apache Cloudberry on
+# Rocky Linux 9. It ensures artifact integrity, performs installation
+# tests, validates key operations, and provides detailed test reports,
+# including handling for ignored test cases.
#
-# Workflow Structure:
-# 1. Skip Check:
-# - Evaluates CI skip flags in commit messages or PR titles
-# - Supports [skip ci], [ci skip], [no ci]
-# - Successfully completes all checks if skip detected
+# Workflow Overview:
+# 1. **Check Skip**:
+# - Dynamically determines if the workflow should run based on CI skip flags.
+# - Evaluates the following fields for skip flags:
+# - **Pull Request Events**: PR title and PR body.
+# - **Push Events**: Commit message of the head commit.
+# - Supports the following skip patterns (case-insensitive):
+# - `[skip ci]`
Review Comment:
> Thanks for supporting this cool feature. One question, code modification and final merging require running through CI at least once, but submitting code during review can skip CI, right?
@my-ship-it - Yes, that's correct. For changes that don't affect the Cloudberry source code (like documentation updates), you can skip the CI pipeline by adding any of these tags to your PR title or body:
```
[skip ci]
[ci skip]
[no ci]
```
However, as you noted, code changes and merging will still require passing through CI for quality assurance. This feature will be particularly useful for our upcoming Apache incubation-related PRs, allowing us to move more efficiently with non-code changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Adding additional test suites. [cloudberry]
Posted by "edespino (via GitHub)" <gi...@apache.org>.
edespino commented on PR #735:
URL: https://github.com/apache/cloudberry/pull/735#issuecomment-2505201490
> reliable.
I will enable a lot more soon. I'm trying to keep CI times down. I have a few ideas on how to approach it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Adding additional test suite. [cloudberry]
Posted by "yjhjstz (via GitHub)" <gi...@apache.org>.
yjhjstz commented on PR #735:
URL: https://github.com/apache/cloudberry/pull/735#issuecomment-2510738699
@foreyes help take a look ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Adding additional test suites. [cloudberry]
Posted by "my-ship-it (via GitHub)" <gi...@apache.org>.
my-ship-it commented on code in PR #735:
URL: https://github.com/apache/cloudberry/pull/735#discussion_r1860411888
##########
.github/workflows/build-cloudberry.yml:
##########
@@ -17,102 +17,86 @@
# permissions and limitations under the License.
#
# --------------------------------------------------------------------
+# GitHub Actions Workflow: Apache Cloudberry Build Pipeline
+# --------------------------------------------------------------------
+# Description:
#
-# GitHub Actions Workflow: Apache Cloudberry Build
-# Description: Builds, tests, and packages Apache Cloudberry on Rocky Linux 9
+# This workflow builds, tests, and packages Apache Cloudberry on
+# Rocky Linux 9. It ensures artifact integrity, performs installation
+# tests, validates key operations, and provides detailed test reports,
+# including handling for ignored test cases.
#
-# Workflow Structure:
-# 1. Skip Check:
-# - Evaluates CI skip flags in commit messages or PR titles
-# - Supports [skip ci], [ci skip], [no ci]
-# - Successfully completes all checks if skip detected
+# Workflow Overview:
+# 1. **Check Skip**:
+# - Dynamically determines if the workflow should run based on CI skip flags.
+# - Evaluates the following fields for skip flags:
+# - **Pull Request Events**: PR title and PR body.
+# - **Push Events**: Commit message of the head commit.
+# - Supports the following skip patterns (case-insensitive):
+# - `[skip ci]`
Review Comment:
Thanks for supporting this cool feature. One question, code modification and final merging require running through CI at least once, but submitting code during review can skip CI, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Adding additional test suites. [cloudberry]
Posted by "edespino (via GitHub)" <gi...@apache.org>.
edespino commented on code in PR #735:
URL: https://github.com/apache/cloudberry/pull/735#discussion_r1861857222
##########
.asf.yaml:
##########
@@ -88,6 +88,9 @@ github:
- RPM Install Test Apache Cloudberry
- ic-good-opt-off
- ic-expandshrink
+ - ic-singlenode_isolation
+ - ic-singlenode_isolation2
+ - ic-resgroup-v2
Review Comment:
> hi ed,
>
> I don't quite understand how you turned on cgroup-v2？
>
> Because the image we run on github CI is opened with CGroup V1 (you can only choose one of the two in linux kernel). And I asked relevant colleagues before, we have to use the same CI image to running CI?
@jiaqizho Let me explain the cgroups v2 setup in our current CI environment.
We've made significant changes to our CI infrastructure. Previously, we used GitHub self-hosted runners with manually created Rocky Linux 8 VM images in AWS, which required substantial infrastructure maintenance. Now, we're using GitHub-hosted custom containers running Rocky Linux 9, which simplifies our setup considerably.
Our container configurations live in https://github.com/apache/cloudberry-devops-release/tree/main/images/docker/cbdb, and they're built through workflows at https://github.com/apache/cloudberry-devops-release/actions. These containers come with all necessary build and runtime packages pre-installed.
Regarding cgroups specifically:
- In the old Rocky Linux 8 environment, we used cgroups v1, configured via https://github.com/apache/cloudberry/blob/main/concourse/scripts/ic_gpdb_resgroup.bash
- With Rocky Linux 9, we switched to cgroups v2 (the current standard), previously configured using https://github.com/apache/cloudberry/blob/main/concourse/scripts/ic_gpdb_resgroup_v2.bash
The current workflow enables cgroups v2 through two main steps:
1. Starting the container with specific cgroups v2 support options
2. Running a "Setup cgroups" step that configures cgroups v2 for testing (based on enhanced logic from the original ic_gpdb_resgroup_v2.bash script)
For testing, we're using the installcheck-resgroup-v2 configuration. Note that I had to update the resgroup_memory_limit_resgroup_v2.out test output file to account for differences between cgroups v1 and v2 environments.
If it would be helpful, I'd be happy to schedule a video call to walk through this setup in more detail.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Adding additional test suites. [cloudberry]
Posted by "gfphoenix78 (via GitHub)" <gi...@apache.org>.
gfphoenix78 commented on PR #735:
URL: https://github.com/apache/cloudberry/pull/735#issuecomment-2505198891
Awesome! These test suits will make the cloudberry more stable and reliable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Adding additional test suites. [cloudberry]
Posted by "jiaqizho (via GitHub)" <gi...@apache.org>.
jiaqizho commented on code in PR #735:
URL: https://github.com/apache/cloudberry/pull/735#discussion_r1861782081
##########
.asf.yaml:
##########
@@ -88,6 +88,9 @@ github:
- RPM Install Test Apache Cloudberry
- ic-good-opt-off
- ic-expandshrink
+ - ic-singlenode_isolation
+ - ic-singlenode_isolation2
+ - ic-resgroup-v2
Review Comment:
hi ed,
I don't quite understand how you turned on cgroup-v2？
Because the image we run on github CI is opened with CGroup V1 (you can only choose one of the two in linux kernel). And I asked relevant colleagues before, we have to use the same CI image to running CI?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Adding additional test suite. [cloudberry]
Posted by "gfphoenix78 (via GitHub)" <gi...@apache.org>.
gfphoenix78 merged PR #735:
URL: https://github.com/apache/cloudberry/pull/735
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org