You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/07/15 13:05:06 UTC
[GitHub] [arrow] assignUser opened a new pull request, #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
assignUser opened a new pull request, #13622:
URL: https://github.com/apache/arrow/pull/13622
@nealrichardson This should be it, though it is hard to test end2end.
@kou Thanks for your suggestions, please look over my other changes to the process.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
assignUser commented on PR #13622:
URL: https://github.com/apache/arrow/pull/13622#issuecomment-1189570969
@github-actions crossbow submit r-binary-packages test-r-versions test-r-offline-minimal test-r-linux-as-cran
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on PR #13622:
URL: https://github.com/apache/arrow/pull/13622#issuecomment-1189042099
This looks right in principle to me. If you're done with changes, you should also run `r-binary-packages`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on a diff in pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
assignUser commented on code in PR #13622:
URL: https://github.com/apache/arrow/pull/13622#discussion_r923578405
##########
r/tools/nixlibs.R:
##########
@@ -22,7 +22,14 @@ dst_dir <- paste0("libarrow/arrow-", VERSION)
# TESTING is set in test-nixlibs.R; it won't be set when called from configure
test_mode <- exists("TESTING")
-arrow_repo <- paste0(getOption("arrow.dev_repo", "https://nightlies.apache.org/arrow/r"), "/libarrow/")
+# Check if version string has 4th component
Review Comment:
Ah, should we harmonize the behavior between nix and windows as winlibs already has this? I think it makes sense to use your proposed idea for both as going forward all releases will have binaries uploaded to the artifactory. (So no cutting off patch versions etc.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13622:
URL: https://github.com/apache/arrow/pull/13622#issuecomment-1189571683
Revision: e0fc21814d244d7d92d8b738cb677fc6e4049406
Submitted crossbow builds: [ursacomputing/crossbow @ actions-43164aaf14](https://github.com/ursacomputing/crossbow/branches/all?query=actions-43164aaf14)
|Task|Status|
|----|------|
|r-binary-packages|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-43164aaf14-github-r-binary-packages)|
|test-r-linux-as-cran|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-43164aaf14-github-test-r-linux-as-cran)|
|test-r-offline-minimal|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-43164aaf14-azure-test-r-offline-minimal)|
|test-r-versions|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-43164aaf14-github-test-r-versions)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on a diff in pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
assignUser commented on code in PR #13622:
URL: https://github.com/apache/arrow/pull/13622#discussion_r923169224
##########
dev/release/05-binary-upload.sh:
##########
@@ -104,6 +105,28 @@ fi
if [ ${UPLOAD_PYTHON} -gt 0 ]; then
rake_tasks+=(python:rc)
fi
+if [ ${UPLOAD_R} -gt 0 ]; then
+ rake_tasks+=(r:rc)
+ # The R libarrow binaries have to be converted to the correct dir structure
+ pushd $artifact_dir/r-binary-packages
+ # R package binaries are distributed via CRAN
+ rm r-pkg*
+ r_src_files=$(ls r-lib*)
+ # Decode relative path from file name.
+ # r-lib__libarrow__bin__centos-7__arrow-8.0.0.zip
+ # --> libarrow/bin/centos-7/arrow-8.0.0.zip
+ r_dest_files=($(echo $r_src_files | sed 's/r-lib__//g; s/__/\//g'))
+ r_src_files=($r_src_files)
+
+ mkdir -p $(dirname ${r_dest_files[@]})
+ r_length=${#r_dest_files[@]}
+
+ for (( i=0; i<${r_length}; i++ ));
+ do
+ mv ${r_src_files[$i]} ${r_dest_files[$i]}
+ done
+ popd
Review Comment:
Ok, I could also use `cp` and add a clean up step in the beginning? That way it would still be repeatable, but feel free to add a diff for the ruby version (or add it in a follow up?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou merged pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
kou merged PR #13622:
URL: https://github.com/apache/arrow/pull/13622
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on a diff in pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on code in PR #13622:
URL: https://github.com/apache/arrow/pull/13622#discussion_r923428064
##########
r/tools/winlibs.R:
##########
@@ -37,28 +37,32 @@ if (!file.exists(sprintf("windows/arrow-%s/include/arrow/api.h", VERSION))) {
silent = quietly
)
}
+
# URL templates
nightly <- paste0(
getOption("arrow.dev_repo", "https://nightlies.apache.org/arrow/r"),
"/libarrow/bin/windows/arrow-%s.zip"
)
- rwinlib <- "https://github.com/rwinlib/arrow/archive/v%s.zip"
Review Comment:
Yeah I think we want to keep rwinlib as the first check, it makes sense for us to stay plugged into that project even if it's not completely necessary anymore.
##########
r/tools/nixlibs.R:
##########
@@ -22,7 +22,14 @@ dst_dir <- paste0("libarrow/arrow-", VERSION)
# TESTING is set in test-nixlibs.R; it won't be set when called from configure
test_mode <- exists("TESTING")
-arrow_repo <- paste0(getOption("arrow.dev_repo", "https://nightlies.apache.org/arrow/r"), "/libarrow/")
+# Check if version string has 4th component
Review Comment:
There's one other case that we should support here. Suppose we submit 9.0.0 to CRAN and they reject it (as happens about 50% of the time). Assume the fix is something in the R package only. We tweak that and resubmit as 9.0.0.1. We want the official artifactory 9.0.0 libarrow to be used.
Here's some old code that did some inspection of the 4th component and did different things: https://github.com/apache/arrow/blob/6c7c4f0a09a3f40d65999511c77d0f21473b4de8/r/tools/nixlibs.R#L216-L233 We don't want exactly that, but maybe something that downloads from artifactory if `is.na(p) || p < 100` or something like that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on a diff in pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
assignUser commented on code in PR #13622:
URL: https://github.com/apache/arrow/pull/13622#discussion_r923313027
##########
r/tools/winlibs.R:
##########
@@ -37,28 +37,32 @@ if (!file.exists(sprintf("windows/arrow-%s/include/arrow/api.h", VERSION))) {
silent = quietly
)
}
+
# URL templates
nightly <- paste0(
getOption("arrow.dev_repo", "https://nightlies.apache.org/arrow/r"),
"/libarrow/bin/windows/arrow-%s.zip"
)
- rwinlib <- "https://github.com/rwinlib/arrow/archive/v%s.zip"
Review Comment:
@nealrichardson let me know if you want to keep rwinlib and add the artifactory as an additional step
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
assignUser commented on PR #13622:
URL: https://github.com/apache/arrow/pull/13622#issuecomment-1189683026
@nealrichardson most of the failing crossbow jobs were unrelated issues that were fixed by rebasing, so I think this should be ready.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on a diff in pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on code in PR #13622:
URL: https://github.com/apache/arrow/pull/13622#discussion_r923642856
##########
r/tools/nixlibs.R:
##########
@@ -22,7 +22,14 @@ dst_dir <- paste0("libarrow/arrow-", VERSION)
# TESTING is set in test-nixlibs.R; it won't be set when called from configure
test_mode <- exists("TESTING")
-arrow_repo <- paste0(getOption("arrow.dev_repo", "https://nightlies.apache.org/arrow/r"), "/libarrow/")
+# Check if version string has 4th component
Review Comment:
I think the winlibs logic as it is isn't quite right for this. I agree they should probably be the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on a diff in pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
assignUser commented on code in PR #13622:
URL: https://github.com/apache/arrow/pull/13622#discussion_r923578405
##########
r/tools/nixlibs.R:
##########
@@ -22,7 +22,14 @@ dst_dir <- paste0("libarrow/arrow-", VERSION)
# TESTING is set in test-nixlibs.R; it won't be set when called from configure
test_mode <- exists("TESTING")
-arrow_repo <- paste0(getOption("arrow.dev_repo", "https://nightlies.apache.org/arrow/r"), "/libarrow/")
+# Check if version string has 4th component
Review Comment:
Ah, should we harmonize the behavior between nix and windows as winlibs already has this? I think it makes sense to use you proposed idea for both as going forward all releases will have binaries uploaded to the artifactory. (So no cutting off patch versions etc.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
kou commented on PR #13622:
URL: https://github.com/apache/arrow/pull/13622#issuecomment-1191006292
Objectives of them are different:
Tests in the `r-binary-packages` task is for CI.
`dev/release/verify-release-candidate*` is for voting by PMC members.
See also: https://www.apache.org/legal/release-policy.html#release-approval
> Before casting +1 binding votes, individuals are REQUIRED to download all signed source code packages onto their own hardware, verify that they meet all requirements of ASF policy on releases as described below, validate all cryptographic signatures, compile as provided, and test the result on their own platform.
For `dev/release/verify-release-candidate*`, it's better that R tests are run on a PMC member's environment. It may not be covered by the `r-binary-packages` task. For example, I'm using Debian GNU/Linux sid but the environment doesn't exist in the `r-binary-packages` task.
Anyway, this can be a follow up task as you said.
@nealrichardson Do you want to take a look at this again before we merge this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
assignUser commented on PR #13622:
URL: https://github.com/apache/arrow/pull/13622#issuecomment-1190384611
@kou Thank you for the ruby changes! I have tested the urls with `nixlibs.R` and `winlibs.R` and they work, so the dir structure is correct and the additional files are no problem.
I am not sure how we would verify the binaries in `dev/release/verify-release-candidate*` in a meaningful way as we already verify them against windows (2 R versions) and 8 linux images in the `r-binary-packages` task. I am open for suggestions though but do think this should be a follow up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
assignUser commented on PR #13622:
URL: https://github.com/apache/arrow/pull/13622#issuecomment-1189055456
@github-actions crossbow submit -g r
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on a diff in pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
assignUser commented on code in PR #13622:
URL: https://github.com/apache/arrow/pull/13622#discussion_r923297609
##########
dev/release/05-binary-upload.sh:
##########
@@ -104,6 +105,28 @@ fi
if [ ${UPLOAD_PYTHON} -gt 0 ]; then
rake_tasks+=(python:rc)
fi
+if [ ${UPLOAD_R} -gt 0 ]; then
+ rake_tasks+=(r:rc)
+ # The R libarrow binaries have to be converted to the correct dir structure
+ pushd $artifact_dir/r-binary-packages
+ # R package binaries are distributed via CRAN
+ rm r-pkg*
+ r_src_files=$(ls r-lib*)
+ # Decode relative path from file name.
+ # r-lib__libarrow__bin__centos-7__arrow-8.0.0.zip
+ # --> libarrow/bin/centos-7/arrow-8.0.0.zip
+ r_dest_files=($(echo $r_src_files | sed 's/r-lib__//g; s/__/\//g'))
+ r_src_files=($r_src_files)
+
+ mkdir -p $(dirname ${r_dest_files[@]})
+ r_length=${#r_dest_files[@]}
+
+ for (( i=0; i<${r_length}; i++ ));
+ do
+ mv ${r_src_files[$i]} ${r_dest_files[$i]}
+ done
+ popd
Review Comment:
I have added this for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #13622:
URL: https://github.com/apache/arrow/pull/13622#issuecomment-1192421161
Benchmark runs are scheduled for baseline = 8f880880e721d138b2667b66632217a6ce2da581 and contender = ed3e55c4f4ad71b21f298acd250dac2145d093c3. ed3e55c4f4ad71b21f298acd250dac2145d093c3 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Failed :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/a3bc183d20644e8bb56c80f213944753...8bb9aa14e56149828b754f5362fe2733/)
[Failed :arrow_down:0.28% :arrow_up:0.03%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/5230b20ba3dc4e5e8efb2ceef65bedc0...4903d2ead62047198ca7807383fe7d56/)
[Failed :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/dd839fbd080b4cda800897c5b1e33378...85a1844a2f6547edbe2735f019361fdf/)
[Finished :arrow_down:0.18% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/095c49aaa59643c68b453817a9dac90a...a39fd2aa818d46dda23329850f9e034a/)
Buildkite builds:
[Failed] [`ed3e55c4` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/1164)
[Failed] [`ed3e55c4` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/1170)
[Failed] [`ed3e55c4` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/1157)
[Finished] [`ed3e55c4` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/1181)
[Failed] [`8f880880` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/1163)
[Failed] [`8f880880` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/1168)
[Failed] [`8f880880` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/1156)
[Finished] [`8f880880` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/1180)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13622:
URL: https://github.com/apache/arrow/pull/13622#issuecomment-1185529970
https://issues.apache.org/jira/browse/ARROW-17001
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
kou commented on PR #13622:
URL: https://github.com/apache/arrow/pull/13622#issuecomment-1189876717
@assignUser I've moved the implementation to `binary-task.rb` and uploaded nightly-packaging-2022-07-19-0 to:
* https://apache.jfrog.io/artifactory/arrow/r-staging-rc/8.0.0-rc1
* https://apache.jfrog.io/artifactory/arrow/r-staging/8.0.0
Could you confirm them?
BTW, can we verify binaries at https://apache.jfrog.io/artifactory/arrow/r-rc/X.Y.Z-rcN by `dev/release/verify-release-candidate.sh` (and `dev/release/verify-release-candidate.bat`)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a diff in pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
kou commented on code in PR #13622:
URL: https://github.com/apache/arrow/pull/13622#discussion_r922518037
##########
r/tools/nixlibs.R:
##########
@@ -22,7 +22,14 @@ dst_dir <- paste0("libarrow/arrow-", VERSION)
# TESTING is set in test-nixlibs.R; it won't be set when called from configure
test_mode <- exists("TESTING")
-arrow_repo <- paste0(getOption("arrow.dev_repo", "https://nightlies.apache.org/arrow/r"), "/libarrow/")
+# Check if version string has 4th component
+is_dev_version <- grepl("(\\d+\\.){3}\\d+", VERSION)
+
+if(is_dev_version){
Review Comment:
```suggestion
if (is_dev_version) {
```
##########
dev/release/05-binary-upload.sh:
##########
@@ -104,6 +105,28 @@ fi
if [ ${UPLOAD_PYTHON} -gt 0 ]; then
rake_tasks+=(python:rc)
fi
+if [ ${UPLOAD_R} -gt 0 ]; then
+ rake_tasks+=(r:rc)
+ # The R libarrow binaries have to be converted to the correct dir structure
+ pushd $artifact_dir/r-binary-packages
+ # R package binaries are distributed via CRAN
+ rm r-pkg*
+ r_src_files=$(ls r-lib*)
+ # Decode relative path from file name.
+ # r-lib__libarrow__bin__centos-7__arrow-8.0.0.zip
+ # --> libarrow/bin/centos-7/arrow-8.0.0.zip
+ r_dest_files=($(echo $r_src_files | sed 's/r-lib__//g; s/__/\//g'))
+ r_src_files=($r_src_files)
+
+ mkdir -p $(dirname ${r_dest_files[@]})
+ r_length=${#r_dest_files[@]}
+
+ for (( i=0; i<${r_length}; i++ ));
+ do
+ mv ${r_src_files[$i]} ${r_dest_files[$i]}
+ done
+ popd
Review Comment:
I want to move this logic to `binary-task.rb`. If we do this at here, we can't run this script multiple times.
I'll provide a diff for it later.
##########
r/tools/winlibs.R:
##########
@@ -37,11 +37,20 @@ if (!file.exists(sprintf("windows/arrow-%s/include/arrow/api.h", VERSION))) {
silent = quietly
)
}
+
+ # Check if version string has 4th component
+ is_dev_version <- grepl("(\\d+\\.){3}\\d+", VERSION)
# URL templates
- nightly <- paste0(
- getOption("arrow.dev_repo", "https://nightlies.apache.org/arrow/r"),
- "/libarrow/bin/windows/arrow-%s.zip"
- )
+ if (is_dev_version) {
+ nightly <- paste0(
+ getOption("arrow.dev_repo", "https://nightlies.apache.org/arrow/r"),
+ "/libarrow/bin/windows/arrow-%s.zip"
+ )
+ } else {
+ # %1$s uses the first variable for both substitutions
+ nightly <- "https://apache.jfrog.io/artifactory/arrow/r/%1$s/libarrow/bin/windows/arrow-%1$s.zip"
Review Comment:
I think that this is not a nightly. Can we use more suitable variable name?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13622:
URL: https://github.com/apache/arrow/pull/13622#issuecomment-1189056734
Revision: 63b3c94ccc47077da469a14aac01e2b7f1c45919
Submitted crossbow builds: [ursacomputing/crossbow @ actions-40974786a5](https://github.com/ursacomputing/crossbow/branches/all?query=actions-40974786a5)
|Task|Status|
|----|------|
|conda-linux-gcc-py37-cpu-r40|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-conda-linux-gcc-py37-cpu-r40)|
|conda-linux-gcc-py37-cpu-r41|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-conda-linux-gcc-py37-cpu-r41)|
|conda-osx-clang-py37-r40|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-conda-osx-clang-py37-r40)|
|conda-osx-clang-py37-r41|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-conda-osx-clang-py37-r41)|
|conda-win-vs2017-py37-r40|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-conda-win-vs2017-py37-r40)|
|conda-win-vs2017-py37-r41|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-conda-win-vs2017-py37-r41)|
|homebrew-r-autobrew|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-40974786a5-github-homebrew-r-autobrew)|
|homebrew-r-brew|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-40974786a5-github-homebrew-r-brew)|
|r-binary-packages|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-40974786a5-github-r-binary-packages)|
|test-fedora-r-clang-sanitizer|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-test-fedora-r-clang-sanitizer)|
|test-r-arrow-backwards-compatibility|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-40974786a5-github-test-r-arrow-backwards-compatibility)|
|test-r-depsource-bundled|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-test-r-depsource-bundled)|
|test-r-depsource-system|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-40974786a5-github-test-r-depsource-system)|
|test-r-dev-duckdb|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-40974786a5-github-test-r-dev-duckdb)|
|test-r-devdocs|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-40974786a5-github-test-r-devdocs)|
|test-r-gcc-11|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-40974786a5-github-test-r-gcc-11)|
|test-r-gcc-12|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-40974786a5-github-test-r-gcc-12)|
|test-r-install-local|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-40974786a5-github-test-r-install-local)|
|test-r-linux-as-cran|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-40974786a5-github-test-r-linux-as-cran)|
|test-r-linux-rchk|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-40974786a5-github-test-r-linux-rchk)|
|test-r-linux-valgrind|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-test-r-linux-valgrind)|
|test-r-minimal-build|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-test-r-minimal-build)|
|test-r-offline-maximal|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-40974786a5-github-test-r-offline-maximal)|
|test-r-offline-minimal|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-test-r-offline-minimal)|
|test-r-rhub-debian-gcc-devel-lto-latest|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-test-r-rhub-debian-gcc-devel-lto-latest)|
|test-r-rhub-debian-gcc-release-custom-ccache|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-test-r-rhub-debian-gcc-release-custom-ccache)|
|test-r-rhub-ubuntu-gcc-release-latest|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-test-r-rhub-ubuntu-gcc-release-latest)|
|test-r-rocker-r-base-latest|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-test-r-rocker-r-base-latest)|
|test-r-rstudio-r-base-4.1-opensuse153|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-test-r-rstudio-r-base-4.1-opensuse153)|
|test-r-rstudio-r-base-4.2-centos7-devtoolset-8|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-test-r-rstudio-r-base-4.2-centos7-devtoolset-8)|
|test-r-rstudio-r-base-4.2-focal|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-test-r-rstudio-r-base-4.2-focal)|
|test-r-ubuntu-22.04|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-40974786a5-github-test-r-ubuntu-22.04)|
|test-r-versions|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-40974786a5-github-test-r-versions)|
|test-ubuntu-18.04-r-sanitizer|[](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-40974786a5-azure-test-ubuntu-18.04-r-sanitizer)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on pull request #13622: ARROW-17001: [Release][R] Use apache artifactory for libarrow binaries.
Posted by GitBox <gi...@apache.org>.
assignUser commented on PR #13622:
URL: https://github.com/apache/arrow/pull/13622#issuecomment-1197966113
@kou I added [ARROW-17236](https://issues.apache.org/jira/browse/ARROW-17236) for the verification.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org