[jira] [Created] (BIGTOP-3443) Building Spark fails on Ubuntu 16.04

["Kengo Seki (Jira)" <ji...@apache.org>]

Kengo Seki created BIGTOP-3443:
----------------------------------

             Summary: Building Spark fails on Ubuntu 16.04
                 Key: BIGTOP-3443
                 URL: https://issues.apache.org/jira/browse/BIGTOP-3443
             Project: Bigtop
          Issue Type: Bug
          Components: spark
            Reporter: Kengo Seki
             Fix For: 1.5.0


On Ubuntu 16.04 (both x86_64 and amd64), building Spark is failing with the following error.
https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/665/COMPONENTS=spark,OS=ubuntu-16.04/console
https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/665/COMPONENTS=spark,OS=ubuntu-16.04-aarch64/console

{code}
* building 'SparkR_2.4.5.tar.gz'

+ find pkg/vignettes/. -not -name . -not -name '*.Rmd' -not -name '*.md' -not -name '*.pdf' -not -name '*.html' -delete
++ grep Version /bigtop/output/spark/spark-core-2.4.5/R/pkg/DESCRIPTION
++ awk '{print $NF}'
+ VERSION=2.4.5
+ CRAN_CHECK_OPTIONS=
+ '[' -n 1 ']'
+ CRAN_CHECK_OPTIONS=' --no-tests'
+ '[' -n '' ']'
+ echo 'Running CRAN check with  --no-tests options'
Running CRAN check with  --no-tests options
+ '[' -n 1 ']'
+ '[' -n '' ']'
+ SPARK_HOME=/bigtop/output/spark/spark-core-2.4.5
+ /usr/bin/R CMD check --no-tests SparkR_2.4.5.tar.gz
* using log directory '/bigtop/output/spark/spark-core-2.4.5/R/SparkR.Rcheck'
* using R version 3.2.3 (2015-12-10)
* using platform: x86_64-pc-linux-gnu (64-bit)
* using session charset: ASCII
* using option '--no-tests'
* checking for file 'SparkR/DESCRIPTION' ... OK
* checking extension type ... Package
* this is package 'SparkR' version '2.4.5'
* checking package namespace information ... OK
* checking package dependencies ... ERROR
Packages suggested but not available: 'rmarkdown' 'testthat'

The suggested packages are required for a complete check.
Checking can be attempted without them by setting the environment
debuild: fatal error at line 1376:
variable _R_CHECK_FORCE_SUGGESTS_ to a false value.

See section 'The DESCRIPTION file' in the 'Writing R Extensions'
dpkg-buildpackage -rfakeroot -D -us -uc -b
 failed
manual.
* DONE

Status: 1 ERROR
{code}

I'm not so familiar with R, but the Docker images of this distro don't seem to have the above R packages.

{code}
sekikn@sekikn-ThinkCentre-M715q:~$ docker rmi -f bigtop/slaves:trunk-ubuntu-16.04
Untagged: bigtop/slaves:trunk-ubuntu-16.04
Untagged: bigtop/slaves@sha256:a7d328f7c7b1d5da757678604e041b36b9b767174f2de298f370524ac475579f
sekikn@sekikn-ThinkCentre-M715q:~$ docker run -it -d bigtop/slaves:trunk-ubuntu-16.04
Unable to find image 'bigtop/slaves:trunk-ubuntu-16.04' locally
trunk-ubuntu-16.04: Pulling from bigtop/slaves
3b37166ec614: Already exists 
ba077e1ddb3a: Already exists 
34c83d2bc656: Already exists 
84b69b6e4743: Already exists 
0f72e97e1f61: Already exists 
dc3135d9e0b7: Already exists 
1bcf2660fa00: Already exists 
62f6bc0949cb: Pull complete 
9e6928b28896: Pull complete 
db781464e36b: Pull complete 
9e1a98618ac7: Pull complete 
Digest: sha256:362379880480575bc4a09bdb8052e01cd364a307df7282165043981752fa7b6f
Status: Downloaded newer image for bigtop/slaves:trunk-ubuntu-16.04
576a624e3e9600bee5ad98ebc8f75525dd98d86c81e2155b1df5eb0c34be66d9
sekikn@sekikn-ThinkCentre-M715q:~$ docker exec -it 576a624e3e96 bash
root@576a624e3e96:/# R -q -e 'packageVersion("rmarkdown")'
> packageVersion("rmarkdown")
Error in packageVersion("rmarkdown") : package 'rmarkdown' not found
Execution halted
root@576a624e3e96:/# R -q -e 'packageVersion("testthat")'
> packageVersion("testthat")
Error in packageVersion("testthat") : package 'testthat' not found
Execution halted
{code}

But building these images succeeded with no error on Jenkins:
https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/156/OS=ubuntu-16.04/console

{code}

(snip)

Notice: /Stage[main]/Bigtop_toolchain::Renv/Exec[install_r_packages]/returns: executed successfully

(snip)

BUILD SUCCESSFUL in 22m 17s
1 actionable task: 1 executed
+ docker push bigtop/slaves:trunk-ubuntu-16.04
The push refers to repository [docker.io/bigtop/slaves]
459ec69fe443: Preparing
694145ca4ac1: Preparing
eb1493b87b0e: Preparing
393364419078: Preparing
36855eaee39c: Preparing
79560c66832e: Preparing
d7232280c8c4: Preparing
07663827a77f: Preparing
87a2d0000622: Preparing
4a7a5ec0f29e: Preparing
8823818c4748: Preparing
d7232280c8c4: Waiting
07663827a77f: Waiting
87a2d0000622: Waiting
8823818c4748: Waiting
4a7a5ec0f29e: Waiting
79560c66832e: Waiting
36855eaee39c: Layer already exists
79560c66832e: Layer already exists
d7232280c8c4: Layer already exists
07663827a77f: Layer already exists
87a2d0000622: Layer already exists
4a7a5ec0f29e: Layer already exists
8823818c4748: Layer already exists
393364419078: Pushed
694145ca4ac1: Pushed
459ec69fe443: Pushed
eb1493b87b0e: Pushed
trunk-ubuntu-16.04: digest: sha256:362379880480575bc4a09bdb8052e01cd364a307df7282165043981752fa7b6f size: 2625
Finished: SUCCESS
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)