You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2022/07/11 07:31:00 UTC
[jira] [Resolved] (SPARK-39735) Enable base image build in lint job and fix sparkr env
[ https://issues.apache.org/jira/browse/SPARK-39735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-39735.
----------------------------------
Fix Version/s: 3.4.0
Resolution: Fixed
Issue resolved by pull request 37149
[https://github.com/apache/spark/pull/37149]
> Enable base image build in lint job and fix sparkr env
> ------------------------------------------------------
>
> Key: SPARK-39735
> URL: https://issues.apache.org/jira/browse/SPARK-39735
> Project: Spark
> Issue Type: Sub-task
> Components: Project Infra
> Affects Versions: 3.4.0
> Reporter: Yikun Jiang
> Assignee: Yikun Jiang
> Priority: Major
> Fix For: 3.4.0
>
>
> Since sparkr 4.2.x has [below new change](https://github.com/r-devel/r-svn/blob/e6be1f6b14838016e78d6a91f48f21acec7fa4c4/doc/NEWS.Rd#L376):
> > Environment variables R_LIBS_USER and R_LIBS_SITE are both now set to the R system default if unset or empty, and can be set to NULL to indicate an empty list of user or site library directories.
> lastest ubuntu pkg also sync this change and has some changes on /etc/R/Renviron ():
> ```
> $ docker run -ti ghcr.io/yikun/apache-spark-github-action-image:sparkr-master-2569799176 cat /etc/R/Renviron | grep R_LIBS_SITE
> R_LIBS_SITE=${R_LIBS_SITE:-'%S'}
> $ docker run -ti ghcr.io/yikun/apache-spark-github-action-image:sparkr-master-2569799176 cat /etc/R/Renviron.site | grep R_LIBS_SITE
> ## edd Jul 2007 Now use R_LIBS_SITE, not R_LIBS
> ## edd Mar 2022 Now in Renviron.site reflecting R_LIBS_SITE
> R_LIBS_SITE="/usr/local/lib/R/site-library/:${R_LIBS_SITE}:/usr/lib/R/library"
> ```
> So, we add `R_LIBS_SITE` to ENV from `/etc/R/Renviron.site` to make sure search paths right for sparkr.
> otherwise, even if we install the `lintr` will cause error like due to `R_LIBS_SITE` wrong set:
> ```
> $ dev/lint-r
> Loading required namespace: SparkR
> Loading required namespace: lintr
> Failed with error: 'there is no package called 'lintr''
> Installing package into '/usr/lib/R/site-library'
> (as 'lib' is unspecified)
> Error in contrib.url(repos, type) :
> trying to use CRAN without setting a mirror
> Calls: install.packages -> startsWith -> contrib.url
> Execution halted
> ```
> [1] https://cran.r-project.org/doc/manuals/r-devel/NEWS.html
> [2] https://stat.ethz.ch/R-manual/R-devel/library/base/html/libPaths.html
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org