You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Neal Richardson (Jira)" <ji...@apache.org> on 2020/05/21 21:42:00 UTC
[jira] [Commented] (ARROW-8880) [R][Linux] Make R Binary Install
Friendlier
[ https://issues.apache.org/jira/browse/ARROW-8880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113564#comment-17113564 ]
Neal Richardson commented on ARROW-8880:
----------------------------------------
It's an interesting idea, though it wouldn't have actually worked in the case of 0.17.1 because the patch release was not built off of master, so arrow-0.17.0.20200518.zip C++ actually would have failed with 0.17.1 R due to ABI incompatibility.
The real problem here is that the 0.17.1 binaries weren't published correctly, it turns out. So I'll fix that. Thanks for the alert.
If you want to try out how your proposal would work, you're welcome to make a PR and we can discuss there. I could see it being interesting for the nightly builds, as they can occasionally fail to build for spurious reasons. I wouldn't want to do it for official releases though--those should match exactly, and should exist.
> [R][Linux] Make R Binary Install Friendlier
> -------------------------------------------
>
> Key: ARROW-8880
> URL: https://issues.apache.org/jira/browse/ARROW-8880
> Project: Apache Arrow
> Issue Type: Improvement
> Components: R
> Affects Versions: 0.17.1
> Environment: Linux (Ubuntu)
> Reporter: Brian Schultheiss
> Priority: Minor
>
> When R install tries to run a binary install, it looks for an exact match on the binary version, say "0.17.1.zip" from [https://dl.bintray.com/ursalabs/arrow-r/libarrow/bin/ubuntu-18.04/].
> The problem is that even though "0.17.1" is pushed to CRAN as an official release, there is a time period (like right now) where bintray does not have an official binary build, just a date stamped build:
>
> arrow-0.17.0.20200516.zip
> arrow-0.17.0.20200517.zip
> arrow-0.17.0.20200518.zip
> arrow-0.17.0.zip
> arrow-0.17.1.20200517.zip
> arrow-0.17.1.20200519.zip
> arrow-0.17.1.20200520.zip
>
> I'd like to suggest adding a new environment variable trigger that would allow for the scanning of bintray for a recent timestamped version, if the specific release number is not present.
> I'd like to suggest enhancing the linux code:
> [https://github.com/apache/arrow/blob/02f7be33d1c32d1636323e6fb90c63cb01bf44af/r/tools/linuxlibs.R#L39-L47]
> with scanning functionality:
> {color:#c1c7d0}try_download <- function(from_url, to_file, scan_dates = FALSE) {{color}
> {color:#c1c7d0} try({color}
> {color:#c1c7d0} suppressWarnings({color}
> {color:#c1c7d0} download.file(from_url, to_file, quiet = quietly){color}
> {color:#c1c7d0} ),{color}
> {color:#c1c7d0} silent = quietly{color}
> {color:#c1c7d0} ){color}
> {color:#0747a6}{color:#c1c7d0} if (!file.exists(to_file)) {{color}{color}
> {color:#0747a6} {color:#c1c7d0} {color}{color:#c1c7d0} if (scan_dates) {{color}{color}
> {color:#0747a6} {color:#0747a6} scan_dates <- format(Sys.Date()-(0:10),"%Y%m%d"){color}{color}
> {color:#0747a6} {color:#0747a6} for (scan_date in scan_dates) {{color}{color}
> {color:#0747a6} {color:#0747a6} base_url <- tools::file_path_sans_ext(from_url){color}{color}
> {color:#0747a6} {color:#0747a6} ext <- tools::file_ext(from_url){color}{color}
> {color:#0747a6} {color:#0747a6} scan_url <- sprintf("%s.%s.%s", base_url, scan_date, ext){color}{color}
> {color:#0747a6} {color:#0747a6} if (try_download(from_url = scan_url, to_file, scan_dates = FALSE)) {{color}{color}
> {color:#0747a6} {color:#0747a6} return(TRUE){color}{color}
> {color:#0747a6} {color:#0747a6} }{color}{color}
> {color:#0747a6} {color:#0747a6} }{color}{color}
> {color:#0747a6} {color:#0747a6} }{color}{color}
> {color:#0747a6} {color:#0747a6} return(FALSE){color}{color}
> {color:#0747a6}{color:#c1c7d0} } else {{color}{color}
> {color:#0747a6} {color:#c1c7d0} return(TRUE){color}{color}
> {color:#0747a6} {color:#c1c7d0} }{color}{color}
> }
> And then augment the calling function:
> [https://github.com/apache/arrow/blob/02f7be33d1c32d1636323e6fb90c63cb01bf44af/r/tools/linuxlibs.R#L55]
>
> with:
> binary_scan_ok <- !identical(tolower(Sys.getenv("LIBARROW_BINARY_SCAN", "false")), "false")
> {color:#c1c7d0}if (try_download(binary_url, libfile, {color}scan_dates = binary_scan_ok{color:#c1c7d0})) {{color}
>
> This would allow automated builds to set the scan option, and then find and install the most recent daily build in lieu of an official binary build being in place.
>
>
>
>
>
>
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)