You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/06/27 15:02:37 UTC
[GitHub] [arrow] nealrichardson opened a new pull request, #13441: ARROW-16912: [R][CI] Fix nightly centos package without GCS
nealrichardson opened a new pull request, #13441:
URL: https://github.com/apache/arrow/pull/13441
cc @assignUser
Most of the diff seems to be my editor trimming whitespace. The actual changes:
* Rename `r-nightly-packages` to `r-binary-packages` since they can be run on demand (not only nightly)
* Add it to the `r` crossbow group
* Turn ARROW_GCS=OFF in the centos-7 package. Where this setting happens is not obvious.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13441: ARROW-16912: [R][CI] Fix nightly centos package without GCS
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13441:
URL: https://github.com/apache/arrow/pull/13441#issuecomment-1167472662
Revision: 96a52e3a4bba56951a94859715de62fe40842eeb
Submitted crossbow builds: [ursacomputing/crossbow @ actions-c9f67fcc1d](https://github.com/ursacomputing/crossbow/branches/all?query=actions-c9f67fcc1d)
|Task|Status|
|----|------|
|r-binary-packages|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-c9f67fcc1d-github-r-binary-packages)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on a diff in pull request #13441: ARROW-16912: [R][CI] Fix nightly centos package without GCS
Posted by GitBox <gi...@apache.org>.
assignUser commented on code in PR #13441:
URL: https://github.com/apache/arrow/pull/13441#discussion_r908318438
##########
docker-compose.yml:
##########
@@ -410,9 +410,9 @@ services:
ARROW_MIMALLOC: "ON"
command: >
/bin/bash -c "
- if grep -q -i -e 'centos.* 7' /etc/os-release; then export ARROW_S3=OFF ARROW_MIMALLOC=OFF; fi &&
+ if grep -q -i -e 'centos.* 7' /etc/os-release; then export ARROW_S3=OFF ARROW_GCS=OFF ARROW_MIMALLOC=OFF; fi &&
Review Comment:
Yeah this service is fixed to centos 7 so we should be fine using `environment:`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on pull request #13441: ARROW-16912: [R][CI] Fix nightly centos package without GCS
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on PR #13441:
URL: https://github.com/apache/arrow/pull/13441#issuecomment-1168109321
> 👎 In regards to adding `r-binary-packages` to the r-group. I think we should take a closer look at the PR CI `r.yml` and see if we want to change it to be more in-line with `r-binary-packages` as they are quite similar instead of adding the task to the r-group. The heavy lifting happens in both jobs already (windows-cpp) and I think `r.yml` should have a macos build anyway.
I don't understand the objection. When I do a PR that affects different build settings in R, I want to run `crossbow submit -g r` and know that I've tested everything. To me, it doesn't make sense for that not to be exhaustive, even if it is somewhat wasteful. Yes, there is some redundancy, but this workflow does different work--clearly, otherwise I would have caught this in ARROW-16510, where I checked (what I thought was) all of the R workflows before merging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on pull request #13441: ARROW-16912: [R][CI] Fix nightly centos package without GCS
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on PR #13441:
URL: https://github.com/apache/arrow/pull/13441#issuecomment-1169404242
> Would it make sense to add one macOS build as that is not covered at all? Or are mac specific failures to rare for that to make sense?
macOS specific failures haven't been a big source of surprises on CI (as in, something gets merged that passes CI but breaks a nightly build). My guess why is (1) many of us working on the R bindings develop on macs so catch them locally and (2) if there's a problem in the C++ library, there are macOS C++ jobs that catch them. Whereas the centos-7 and rtools35 jobs (in particular) test things in the C++ library that aren't covered elsewhere.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on pull request #13441: ARROW-16912: [R][CI] Fix nightly centos package without GCS
Posted by GitBox <gi...@apache.org>.
assignUser commented on PR #13441:
URL: https://github.com/apache/arrow/pull/13441#issuecomment-1168549592
@nealrichardson Maybe I phrased this to strongly. It's not really an objection and should not block this PR. My intention was to start a discussion about/review of the `r.yml` CI and if it does what we want&need or if we should extend it.
You are right, it does make sense that all R related jobs are in the R group to prevent issues like this happening.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on pull request #13441: ARROW-16912: [R][CI] Fix nightly centos package without GCS
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on PR #13441:
URL: https://github.com/apache/arrow/pull/13441#issuecomment-1168645059
> @nealrichardson Maybe I phrased this to strongly. It's not really an objection and should not block this PR. My intention was to start a discussion about/review of the `r.yml` CI and if it does what we want&need or if we should extend it. You are right, it does make sense that all R related jobs are in the R group to prevent issues like this happening.
👍 cool, we can review it. We've selected builds in the r.yml in order to catch the widest spectrum of issues with the fewest builds, but it's always worth evaluating if that's still the case. Currently, that's (1) separate C++ shared library build with everything on, force_tests: true; (2) bundled static build on centos-7/gcc 4.8; (3) windows, all rtools versions (we once pushed the rtools35 build to nightly but it kept failing because people would merge things that passed CI but broke the build, so we moved it back).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson merged pull request #13441: ARROW-16912: [R][CI] Fix nightly centos package without GCS
Posted by GitBox <gi...@apache.org>.
nealrichardson merged PR #13441:
URL: https://github.com/apache/arrow/pull/13441
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on pull request #13441: ARROW-16912: [R][CI] Fix nightly centos package without GCS
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on PR #13441:
URL: https://github.com/apache/arrow/pull/13441#issuecomment-1170402265
@github-actions crossbow submit r-binary-packages
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13441: ARROW-16912: [R][CI] Fix nightly centos package without GCS
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13441:
URL: https://github.com/apache/arrow/pull/13441#issuecomment-1167468870
https://issues.apache.org/jira/browse/ARROW-16912
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on a diff in pull request #13441: ARROW-16912: [R][CI] Fix nightly centos package without GCS
Posted by GitBox <gi...@apache.org>.
assignUser commented on code in PR #13441:
URL: https://github.com/apache/arrow/pull/13441#discussion_r907563449
##########
docker-compose.yml:
##########
@@ -410,9 +410,9 @@ services:
ARROW_MIMALLOC: "ON"
command: >
/bin/bash -c "
- if grep -q -i -e 'centos.* 7' /etc/os-release; then export ARROW_S3=OFF ARROW_MIMALLOC=OFF; fi &&
+ if grep -q -i -e 'centos.* 7' /etc/os-release; then export ARROW_S3=OFF ARROW_GCS=OFF ARROW_MIMALLOC=OFF; fi &&
Review Comment:
On second thought the logic should probably just be removed and the vars set under `environment:`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on pull request #13441: ARROW-16912: [R][CI] Fix nightly centos package without GCS
Posted by GitBox <gi...@apache.org>.
assignUser commented on PR #13441:
URL: https://github.com/apache/arrow/pull/13441#issuecomment-1169336857
With the current dearth of CI resources (atm 16 jobs queued 1 running) my thought of extending the `r.yml` to basically be `r-binary-packages` isn't really feasible.
Would it make sense to add one macOS build as that is not covered at all? Or are mac specific failures to rare for that to make sense?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on pull request #13441: ARROW-16912: [R][CI] Fix nightly centos package without GCS
Posted by GitBox <gi...@apache.org>.
assignUser commented on PR #13441:
URL: https://github.com/apache/arrow/pull/13441#issuecomment-1170214654
> many of us working on macs
Yeah good point. With mac runners being the scarcest it makes sense to skip that build and rely on the nightlies.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on pull request #13441: ARROW-16912: [R][CI] Fix nightly centos package without GCS
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on PR #13441:
URL: https://github.com/apache/arrow/pull/13441#issuecomment-1167466560
@github-actions crossbow submit r-binary-packages
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on a diff in pull request #13441: ARROW-16912: [R][CI] Fix nightly centos package without GCS
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on code in PR #13441:
URL: https://github.com/apache/arrow/pull/13441#discussion_r907940234
##########
docker-compose.yml:
##########
@@ -410,9 +410,9 @@ services:
ARROW_MIMALLOC: "ON"
command: >
/bin/bash -c "
- if grep -q -i -e 'centos.* 7' /etc/os-release; then export ARROW_S3=OFF ARROW_MIMALLOC=OFF; fi &&
+ if grep -q -i -e 'centos.* 7' /etc/os-release; then export ARROW_S3=OFF ARROW_GCS=OFF ARROW_MIMALLOC=OFF; fi &&
Review Comment:
IIUC the intent is that these are only set on centos 7, and you can't conditionally set env vars in `environment:`. Though the setup appears to have baked in centos 7 from where it inherits, so maybe that's not a concern?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13441: ARROW-16912: [R][CI] Fix nightly centos package without GCS
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13441:
URL: https://github.com/apache/arrow/pull/13441#issuecomment-1170403248
Revision: f21066243758a03b841b3e6e9dc899f3b67a0d78
Submitted crossbow builds: [ursacomputing/crossbow @ actions-2b8b31eb5d](https://github.com/ursacomputing/crossbow/branches/all?query=actions-2b8b31eb5d)
|Task|Status|
|----|------|
|r-binary-packages|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-2b8b31eb5d-github-r-binary-packages)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org