You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/07/20 18:28:47 UTC
[GitHub] [arrow-datafusion] alamb opened a new pull request, #2948: Test Remove CI Caching
alamb opened a new pull request, #2948:
URL: https://github.com/apache/arrow-datafusion/pull/2948
# Which issue does this PR close?
Closes https://github.com/apache/arrow-datafusion/issues/2947
# Rationale for this change
CI is failing after running out of space; I am trying to reduce used space.
# What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->
# Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Dandandan commented on a diff in pull request #2948: Remove CI Caching to preserve diskspace
Posted by GitBox <gi...@apache.org>.
Dandandan commented on code in PR #2948:
URL: https://github.com/apache/arrow-datafusion/pull/2948#discussion_r925980503
##########
.github/workflows/rust.yml:
##########
@@ -46,36 +46,29 @@ jobs:
# and thus do not depend on the OS, arch nor rust version.
path: /github/home/.cargo
key: cargo-cache-
- - name: Cache Rust dependencies
Review Comment:
I found this https://github.com/marketplace/actions/rust-cache which does perform some heuristics and cleaning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Dandandan commented on a diff in pull request #2948: Remove CI Caching to preserve diskspace
Posted by GitBox <gi...@apache.org>.
Dandandan commented on code in PR #2948:
URL: https://github.com/apache/arrow-datafusion/pull/2948#discussion_r925975783
##########
.github/workflows/rust.yml:
##########
@@ -46,36 +46,29 @@ jobs:
# and thus do not depend on the OS, arch nor rust version.
path: /github/home/.cargo
key: cargo-cache-
- - name: Cache Rust dependencies
- uses: actions/cache@v3
- with:
- # these represent compiled steps of both dependencies and arrow
- # and thus are specific for a particular OS, arch and rust version.
- path: /github/home/target
- key: ${{ runner.os }}-${{ matrix.arch }}-target-cache-${{ matrix.rust }}-
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
rust-version: ${{ matrix.rust }}
- - name: Build workspace in debug mode
+ - name: Check workspace in debug mode
run: |
- cargo build
+ cargo check
Review Comment:
:👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #2948: Remove CI Caching to preserve diskspace
Posted by GitBox <gi...@apache.org>.
alamb commented on code in PR #2948:
URL: https://github.com/apache/arrow-datafusion/pull/2948#discussion_r925984942
##########
.github/workflows/rust.yml:
##########
@@ -46,36 +46,29 @@ jobs:
# and thus do not depend on the OS, arch nor rust version.
path: /github/home/.cargo
key: cargo-cache-
- - name: Cache Rust dependencies
Review Comment:
```
This cache is automatically keyed by:
the github [job_id](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_id),
the rustc release / host / hash,
the value of some compiler-specific environment variables (eg. RUSTFLAGS, etc), and
a hash of all Cargo.lock / Cargo.toml files found anywhere in the repository (if present).
a hash of all rust-toolchain / rust-toolchain.toml files in the root of the repository (if present).
```
Seems particularly relevant to why the basic cache wasn't working for us
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Dandandan commented on a diff in pull request #2948: Remove CI Caching to preserve diskspace
Posted by GitBox <gi...@apache.org>.
Dandandan commented on code in PR #2948:
URL: https://github.com/apache/arrow-datafusion/pull/2948#discussion_r925975495
##########
.github/workflows/rust.yml:
##########
@@ -46,36 +46,29 @@ jobs:
# and thus do not depend on the OS, arch nor rust version.
path: /github/home/.cargo
key: cargo-cache-
- - name: Cache Rust dependencies
Review Comment:
I wonder if something is wrong with the config 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Dandandan commented on a diff in pull request #2948: Remove CI Caching to preserve diskspace
Posted by GitBox <gi...@apache.org>.
Dandandan commented on code in PR #2948:
URL: https://github.com/apache/arrow-datafusion/pull/2948#discussion_r925984207
##########
.github/workflows/rust.yml:
##########
@@ -46,36 +46,29 @@ jobs:
# and thus do not depend on the OS, arch nor rust version.
path: /github/home/.cargo
key: cargo-cache-
- - name: Cache Rust dependencies
Review Comment:
Maybe what's wrong here is that the `.cargo` die is not cached?
https://doc.rust-lang.org/cargo/guide/cargo-home.html#caching-the-cargo-home-in-ci
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] codecov-commenter commented on pull request #2948: Remove CI Caching to preserve diskspace
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #2948:
URL: https://github.com/apache/arrow-datafusion/pull/2948#issuecomment-1190652715
# [Codecov](https://codecov.io/gh/apache/arrow-datafusion/pull/2948?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#2948](https://codecov.io/gh/apache/arrow-datafusion/pull/2948?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (91c20f3) into [master](https://codecov.io/gh/apache/arrow-datafusion/commit/944ef3d6eeb5e3b31c5561df9b7b8cbdef7367f5?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (944ef3d) will **decrease** coverage by `0.00%`.
> The diff coverage is `76.19%`.
```diff
@@ Coverage Diff @@
## master #2948 +/- ##
==========================================
- Coverage 85.44% 85.43% -0.01%
==========================================
Files 275 275
Lines 49690 49739 +49
==========================================
+ Hits 42458 42495 +37
- Misses 7232 7244 +12
```
| [Impacted Files](https://codecov.io/gh/apache/arrow-datafusion/pull/2948?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [datafusion/common/src/scalar.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/2948/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9jb21tb24vc3JjL3NjYWxhci5ycw==) | `84.80% <ø> (-0.02%)` | :arrow_down: |
| [datafusion/core/tests/sql/information\_schema.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/2948/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9jb3JlL3Rlc3RzL3NxbC9pbmZvcm1hdGlvbl9zY2hlbWEucnM=) | `98.55% <ø> (ø)` | |
| [datafusion/sql/src/planner.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/2948/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9zcWwvc3JjL3BsYW5uZXIucnM=) | `81.26% <0.00%> (ø)` | |
| [datafusion/core/src/execution/context.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/2948/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9jb3JlL3NyYy9leGVjdXRpb24vY29udGV4dC5ycw==) | `77.95% <7.69%> (-1.12%)` | :arrow_down: |
| [datafusion/core/src/catalog/information\_schema.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/2948/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9jb3JlL3NyYy9jYXRhbG9nL2luZm9ybWF0aW9uX3NjaGVtYS5ycw==) | `94.67% <95.91%> (+0.95%)` | :arrow_up: |
| [datafusion/expr/src/window\_frame.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/2948/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9leHByL3NyYy93aW5kb3dfZnJhbWUucnM=) | `92.43% <0.00%> (-0.85%)` | :arrow_down: |
| [datafusion/expr/src/logical\_plan/plan.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/2948/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9leHByL3NyYy9sb2dpY2FsX3BsYW4vcGxhbi5ycw==) | `77.14% <0.00%> (+0.17%)` | :arrow_up: |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] ursabot commented on pull request #2948: Remove CI Caching to preserve diskspace
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #2948:
URL: https://github.com/apache/arrow-datafusion/pull/2948#issuecomment-1190673378
Benchmark runs are scheduled for baseline = d11e28a333992ffe71303a19a4891a354c8af240 and contender = b49093c7428057ec157d14b0808b171804177a49. b49093c7428057ec157d14b0808b171804177a49 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped :warning: Benchmarking of arrow-datafusion-commits is not supported on ec2-t3-xlarge-us-east-2] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/f9f6387808b44f888200e27e8ce61428...5c50ce90fcc8497ebc4fe96b7145654e/)
[Skipped :warning: Benchmarking of arrow-datafusion-commits is not supported on test-mac-arm] [test-mac-arm](https://conbench.ursa.dev/compare/runs/e03024f840114a6983e1f4d5a27a5a1a...694f4b10ed9d48bdaa0257e1ec398099/)
[Skipped :warning: Benchmarking of arrow-datafusion-commits is not supported on ursa-i9-9960x] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/ed7bf83495544cd4986faa0c74c36743...8ff564b908444793b539bda09e48247a/)
[Skipped :warning: Benchmarking of arrow-datafusion-commits is not supported on ursa-thinkcentre-m75q] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/689a07ad6b1948b6bab7f82390415319...33258172a423469ea670bcc02702c411/)
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #2948: Test Remove CI Caching
Posted by GitBox <gi...@apache.org>.
alamb commented on code in PR #2948:
URL: https://github.com/apache/arrow-datafusion/pull/2948#discussion_r925936153
##########
.github/workflows/rust.yml:
##########
@@ -46,36 +46,29 @@ jobs:
# and thus do not depend on the OS, arch nor rust version.
path: /github/home/.cargo
key: cargo-cache-
- - name: Cache Rust dependencies
Review Comment:
I have never been convinced this caching is particularly effective so I am going to test what happens when I remove it...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #2948: Test Remove CI Caching
Posted by GitBox <gi...@apache.org>.
alamb commented on code in PR #2948:
URL: https://github.com/apache/arrow-datafusion/pull/2948#discussion_r925959914
##########
.github/workflows/rust.yml:
##########
@@ -107,12 +100,6 @@ jobs:
path: /github/home/.cargo
# this key equals the ones on `linux-build-lib` for re-use
key: cargo-cache-
- - name: Cache Rust dependencies
Review Comment:
For what it is worth the last time this CI check passed on master was https://github.com/apache/arrow-datafusion/runs/7399254770?check_suite_focus=true where it took 24m and 26 seconds (after the linux-build-lib job finished)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb merged pull request #2948: Remove CI Caching to preserve diskspace
Posted by GitBox <gi...@apache.org>.
alamb merged PR #2948:
URL: https://github.com/apache/arrow-datafusion/pull/2948
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org