You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "Jiashu-Hu (via GitHub)" <gi...@apache.org> on 2025/03/19 17:44:02 UTC
[PR] updatted github action by change version tag to sha hashes [datafusion]
Jiashu-Hu opened a new pull request, #15315:
URL: https://github.com/apache/datafusion/pull/15315
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
-->
- Closes #[15298](https://github.com/apache/datafusion/issues/15298).
## Rationale for this change
This update strengthens the security of GitHub workflows by substituting version tags with precise SHA hashes for actions. This modification reduces the risks tied to supply chain attacks by guaranteeing that only validated action versions are utilized in the workflows. This method adheres to industry best practices for safeguarding CI/CD pipelines.
<!--
Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.
-->
## What changes are included in this PR?
Replaced version tags with specific SHA hashes for actions in the GitHub workflows.
Original:
[dev.yml](https://github.com/apache/datafusion/blob/main/.github/workflows/dev.yml) -> - uses: korandoru/hawkeye@v6
[rust.yml](https://github.com/apache/datafusion/blob/main/.github/workflows/rust.yml) -> - uses: korandoru/hawkeye@v6
[setup-macos-aarch64-builder/action.yaml](https://github.com/apache/datafusion/blob/main/.github/actions/setup-macos-aarch64-builder/action.yaml) -> uses: Swatinem/rust-cache@v2
[setup-rust-runtime/action.yaml](https://github.com/apache/datafusion/blob/main/.github/actions/setup-rust-runtime/action.yaml) -> uses: mozilla-actions/sccache-action@v0.0.4
Update:
[dev.yml](https://github.com/apache/datafusion/blob/main/.github/workflows/dev.yml) -> - uses: korandoru/hawkeye@dd74178a96f27b1121447c6b4a4ccfce180d5bf7
[rust.yml](https://github.com/apache/datafusion/blob/main/.github/workflows/rust.yml) -> - uses: korandoru/hawkeye@dd74178a96f27b1121447c6b4a4ccfce180d5bf7
[setup-macos-aarch64-builder/action.yaml](https://github.com/apache/datafusion/blob/main/.github/actions/setup-macos-aarch64-builder/action.yaml) -> uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6
[setup-rust-runtime/action.yaml](https://github.com/apache/datafusion/blob/main/.github/actions/setup-rust-runtime/action.yaml) -> uses: uses: mozilla-actions/sccache-action@2e7f9ec7921547d4b46598398ca573513895d0bd
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->
## Are these changes tested?
Yes, the changes are tested to ensure that the GitHub workflows function correctly with the specified SHA hashes.
Where I got this Hash:
korandoru/hawkeye@dd74178a96f27b1121447c6b4a4ccfce180d5bf7:
This is implemented in v6 of the original GitHub Action, but it redirects to v6.0.1. Consequently, I’ve bound the hash to v6.0.1 here for consistency and security.
https://github.com/korandoru/hawkeye/commit/dd74178a96f27b1121447c6b4a4ccfce180d5bf7
<img width="655" alt="image" src="https://github.com/user-attachments/assets/f4ce0107-4bd4-42fb-a0b6-11e9b568f6fd" />
Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6:
This is implemented in v2 of the original GitHub Action, but it redirects to v2.7.8. Consequently, I’ve bound the hash to v2.7.8 here for consistency and security.
https://github.com/Swatinem/rust-cache/commit/9d47c6ad4b02e050fd481d890b2ea34778fd09d6
<img width="655" alt="image" src="https://github.com/user-attachments/assets/af1c38df-c034-495a-affb-d9251ade2b19" />
mozilla-actions/sccache-action@2e7f9ec7921547d4b46598398ca573513895d0bd:
https://github.com/mozilla-actions/sccache-action/commit/2e7f9ec7921547d4b46598398ca573513895d0bd
This is implemented in v0.0.4 of the original GitHub Action, so that I just bound the hash to v0.0.4 here for consistency and security.
<img width="659" alt="image" src="https://github.com/user-attachments/assets/098cc6a1-2713-4dfe-8ddf-4f6118eabdfe" />
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
-->
## Are there any user-facing changes?
No, all of that is under github action
<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] updatted github action by change version tag to sha hashes [datafusion]
Posted by "Jiashu-Hu (via GitHub)" <gi...@apache.org>.
Jiashu-Hu commented on PR #15315:
URL: https://github.com/apache/datafusion/pull/15315#issuecomment-2741526839
> Well that is unfortunate. I wonder if the apache regex is correct - the one in the error message is not, should be `.*\/.*@[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]+`
Yes, you're correct— and I've checked that `korandoru/hawkeye@dd74178a96f27b1121447c6b4a4ccfce180d5bf7` does match the regex pattern you provided `.*\/.*@[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]+`, as the commit hash is a valid 40-character hexadecimal string. Since these actions run successfully in your repository, it confirms that the hash values are correct and GitHub can execute them without issues. The error likely arises from specific settings in Apache's GitHub Actions policies or DataFusion's repository configuration, which may restrict the use of commit hashes for external actions.
<img width="1503" alt="image" src="https://github.com/user-attachments/assets/31837bb1-632f-4d30-8e13-c4507a117c21" />
I suppose related setting is in here:
<img width="1482" alt="image" src="https://github.com/user-attachments/assets/ba2ecbdc-ea36-4383-b8b6-92b542cf0421" />
After conducting some research, it seems that the issue arises because mozilla-actions/sccache-action@v0.0.4 has been explicitly added to the allowlist. Once a specific version tag is defined, other versions—including those with commit hashes like mozilla-actions/sccache-action@2e7f9ec7921547d4b46598398ca573513895d0bd—are not matched and thus not permitted.
[Related official document](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository#allowing-select-actions-and-reusable-workflows-to-run)
<img width="756" alt="image" src="https://github.com/user-attachments/assets/26525c8b-acd1-4d80-9e54-e29c40260faf" />
The most effective solution is to either add the specific commit hashes to the allowlist (e.g., mozilla-actions/sccache-action@2e7f9ec7921547d4b46598398ca573513895d0bd) to approve that SHA, or use a wildcard (e.g., mozilla-actions/sccache-action@*) to allow all versions, including both hashes and tags.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] updatted github action by change version tag to sha hashes [datafusion]
Posted by "Jiashu-Hu (via GitHub)" <gi...@apache.org>.
Jiashu-Hu commented on PR #15315:
URL: https://github.com/apache/datafusion/pull/15315#issuecomment-2738308374
hi @alamb, seems there's some regulation to block SHA HASHE value as version tag in apache community
<img width="1092" alt="image" src="https://github.com/user-attachments/assets/ec3357ac-750c-4010-ae11-6455c5429472" />
I tried convert it to a shorter version hash values (7 digit) but it shows
<img width="874" alt="image" src="https://github.com/user-attachments/assets/20afc44a-0628-413e-81e2-d7e2b9e9a1b7" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] updatted github action by change version tag to sha hashes [datafusion]
Posted by "Omega359 (via GitHub)" <gi...@apache.org>.
Omega359 commented on PR #15315:
URL: https://github.com/apache/datafusion/pull/15315#issuecomment-2743064531
I think a committer should bring this up with apache infra structure folks as I cannot see a way to follow the directions in the apache github actions policy here without just yanking those actions.
@alamb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] updatted github action by change version tag to sha hashes [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #15315:
URL: https://github.com/apache/datafusion/pull/15315#issuecomment-2978700211
Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] updatted github action by change version tag to sha hashes [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #15315: updatted github action by change version tag to sha hashes
URL: https://github.com/apache/datafusion/pull/15315
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] updatted github action by change version tag to sha hashes [datafusion]
Posted by "geoffreyclaude (via GitHub)" <gi...@apache.org>.
geoffreyclaude commented on code in PR #15315:
URL: https://github.com/apache/datafusion/pull/15315#discussion_r2049224450
##########
.github/actions/setup-rust-runtime/action.yaml:
##########
@@ -16,23 +16,22 @@
# under the License.
name: Setup Rust Runtime
-description: 'Setup Rust Runtime Environment'
+description: "Setup Rust Runtime Environment"
runs:
using: "composite"
steps:
- name: Run sccache-cache
- uses: mozilla-actions/sccache-action@v0.0.4
+ uses: mozilla-actions/sccache-action@2e7f9ec7921547d4b46598398ca573513895d0bd
- name: Configure runtime env
shell: bash
# do not produce debug symbols to keep memory usage down
# hardcoding other profile params to avoid profile override values
# More on Cargo profiles https://doc.rust-lang.org/cargo/reference/profiles.html?profile-settings#profile-settings
- #
+ #
# Set debuginfo=line-tables-only as debuginfo=0 causes immensely slow build
# See for more details: https://github.com/rust-lang/rust/issues/119560
run: |
echo "RUSTC_WRAPPER=sccache" >> $GITHUB_ENV
echo "SCCACHE_GHA_ENABLED=true" >> $GITHUB_ENV
echo "RUST_BACKTRACE=1" >> $GITHUB_ENV
echo "RUSTFLAGS=-C debuginfo=line-tables-only -C incremental=false" >> $GITHUB_ENV
Review Comment:
The `-C incremental=false` was introduced in https://github.com/apache/datafusion/pull/8730, but I'm pretty sure it should actually be disabled with an env variable instead:
```bash
echo "CARGO_INCREMENTAL=0" >> $GITHUB_ENV
```
The `RUSTCFLAGS` [`incremental`](https://doc.rust-lang.org/rustc/codegen-options/index.html#incremental) only specifies the "path to a directory where incremental files will be stored" — so in this case it is requesting to store incremental files in the folder named "false".
Since sccache does not cache incremental files, using the env var instead of the flag should have some good improvements on the sccache hit rate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] updatted github action by change version tag to sha hashes [datafusion]
Posted by "Omega359 (via GitHub)" <gi...@apache.org>.
Omega359 commented on PR #15315:
URL: https://github.com/apache/datafusion/pull/15315#issuecomment-2741203586
Well that is unfortunate. I wonder if the apache regex is correct - the one in the error message is not, should be '.*\/.*@[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]+'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org