You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/07/20 14:54:11 UTC
[GitHub] [arrow-datafusion] alamb opened a new pull request, #2945: Improve cache in github actions
alamb opened a new pull request, #2945:
URL: https://github.com/apache/arrow-datafusion/pull/2945
# Which issue does this PR close?
N/A
# Rationale for this change
I have seen several PR checks failing recently related to "out of disk space", for example: https://github.com/apache/arrow-datafusion/runs/7430371132?check_suite_focus=true
You can see from that PR that the cargo cache doesn't seem to be very effective (it downloads many crates). It also seems to recompile all the dependencies, despite restoring a cache.
I believe the cache that is being used is not useful (just restoring a bunch of old files that have been outdated for a long time)
# What changes are included in this PR?
1. Change the cache key for `target` to include the git sha (as the point is to restore files from the `linux-build-lib` job earlier)
2. TODO change the cargo cache key likewise so the build job caches the data
# Are there any user-facing changes?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #2945: Improve cache in github actions
Posted by GitBox <gi...@apache.org>.
alamb commented on code in PR #2945:
URL: https://github.com/apache/arrow-datafusion/pull/2945#discussion_r925710401
##########
.github/workflows/rust.yml:
##########
@@ -45,14 +45,14 @@ jobs:
# these represent dependencies downloaded by cargo
# and thus do not depend on the OS, arch nor rust version.
path: /github/home/.cargo
- key: cargo-cache-
+ key: cargo-cache2-
- name: Cache Rust dependencies
uses: actions/cache@v3
with:
# these represent compiled steps of both dependencies and arrow
# and thus are specific for a particular OS, arch and rust version.
path: /github/home/target
- key: ${{ runner.os }}-${{ matrix.arch }}-target-cache-${{ matrix.rust }}-
+ key: ${{ runner.os }}-${{ matrix.arch }}-${{ matrix.rust }}-target-${{ github.sha }}
Review Comment:
This is the core change -- put the sha in to the cache
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on pull request #2945: Improve cache in github actions
Posted by GitBox <gi...@apache.org>.
alamb commented on PR #2945:
URL: https://github.com/apache/arrow-datafusion/pull/2945#issuecomment-1190612313
🤔 it did not appear to help: https://github.com/apache/arrow-datafusion/runs/7432229787?check_suite_focus=true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on pull request #2945: Improve cache in github actions
Posted by GitBox <gi...@apache.org>.
alamb commented on PR #2945:
URL: https://github.com/apache/arrow-datafusion/pull/2945#issuecomment-1190677993
Took a different approach instead: https://github.com/apache/arrow-datafusion/pull/2948
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #2945: Improve cache in github actions
Posted by GitBox <gi...@apache.org>.
alamb commented on code in PR #2945:
URL: https://github.com/apache/arrow-datafusion/pull/2945#discussion_r925711405
##########
.github/workflows/rust.yml:
##########
@@ -473,13 +473,7 @@ jobs:
with:
path: /github/home/.cargo
# this key equals the ones on `linux-build-lib` for re-use
- key: cargo-cache-
- - name: Cache Rust dependencies
Review Comment:
As this job runs `cargo fmt` there is no reason to restore the contents of `target` as part of the job
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb closed pull request #2945: Improve cache in github actions
Posted by GitBox <gi...@apache.org>.
alamb closed pull request #2945: Improve cache in github actions
URL: https://github.com/apache/arrow-datafusion/pull/2945
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org