[jira] [Resolved] (BIGTOP-3443) Building Spark fails on Ubuntu 16.04

["Kengo Seki (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BIGTOP-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kengo Seki resolved BIGTOP-3443.
--------------------------------
    Resolution: Fixed

> Building Spark fails on Ubuntu 16.04
> ------------------------------------
>
>                 Key: BIGTOP-3443
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-3443
>             Project: Bigtop
>          Issue Type: Bug
>          Components: spark
>            Reporter: Kengo Seki
>            Assignee: Yuqi Gu
>            Priority: Major
>             Fix For: 1.5.0
>
>          Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> On Ubuntu 16.04 (both x86_64 and amd64), building Spark is failing with the following error.
> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/665/COMPONENTS=spark,OS=ubuntu-16.04/console
> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/665/COMPONENTS=spark,OS=ubuntu-16.04-aarch64/console
> {code}
> * building 'SparkR_2.4.5.tar.gz'
> + find pkg/vignettes/. -not -name . -not -name '*.Rmd' -not -name '*.md' -not -name '*.pdf' -not -name '*.html' -delete
> ++ grep Version /bigtop/output/spark/spark-core-2.4.5/R/pkg/DESCRIPTION
> ++ awk '{print $NF}'
> + VERSION=2.4.5
> + CRAN_CHECK_OPTIONS=
> + '[' -n 1 ']'
> + CRAN_CHECK_OPTIONS=' --no-tests'
> + '[' -n '' ']'
> + echo 'Running CRAN check with  --no-tests options'
> Running CRAN check with  --no-tests options
> + '[' -n 1 ']'
> + '[' -n '' ']'
> + SPARK_HOME=/bigtop/output/spark/spark-core-2.4.5
> + /usr/bin/R CMD check --no-tests SparkR_2.4.5.tar.gz
> * using log directory '/bigtop/output/spark/spark-core-2.4.5/R/SparkR.Rcheck'
> * using R version 3.2.3 (2015-12-10)
> * using platform: x86_64-pc-linux-gnu (64-bit)
> * using session charset: ASCII
> * using option '--no-tests'
> * checking for file 'SparkR/DESCRIPTION' ... OK
> * checking extension type ... Package
> * this is package 'SparkR' version '2.4.5'
> * checking package namespace information ... OK
> * checking package dependencies ... ERROR
> Packages suggested but not available: 'rmarkdown' 'testthat'
> The suggested packages are required for a complete check.
> Checking can be attempted without them by setting the environment
> debuild: fatal error at line 1376:
> variable _R_CHECK_FORCE_SUGGESTS_ to a false value.
> See section 'The DESCRIPTION file' in the 'Writing R Extensions'
> dpkg-buildpackage -rfakeroot -D -us -uc -b
>  failed
> manual.
> * DONE
> Status: 1 ERROR
> {code}
> I'm not so familiar with R, but the Docker images of this distro don't seem to have the above R packages.
> {code}
> sekikn@sekikn-ThinkCentre-M715q:~$ docker rmi -f bigtop/slaves:trunk-ubuntu-16.04
> Untagged: bigtop/slaves:trunk-ubuntu-16.04
> Untagged: bigtop/slaves@sha256:a7d328f7c7b1d5da757678604e041b36b9b767174f2de298f370524ac475579f
> sekikn@sekikn-ThinkCentre-M715q:~$ docker run -it -d bigtop/slaves:trunk-ubuntu-16.04
> Unable to find image 'bigtop/slaves:trunk-ubuntu-16.04' locally
> trunk-ubuntu-16.04: Pulling from bigtop/slaves
> 3b37166ec614: Already exists 
> ba077e1ddb3a: Already exists 
> 34c83d2bc656: Already exists 
> 84b69b6e4743: Already exists 
> 0f72e97e1f61: Already exists 
> dc3135d9e0b7: Already exists 
> 1bcf2660fa00: Already exists 
> 62f6bc0949cb: Pull complete 
> 9e6928b28896: Pull complete 
> db781464e36b: Pull complete 
> 9e1a98618ac7: Pull complete 
> Digest: sha256:362379880480575bc4a09bdb8052e01cd364a307df7282165043981752fa7b6f
> Status: Downloaded newer image for bigtop/slaves:trunk-ubuntu-16.04
> 576a624e3e9600bee5ad98ebc8f75525dd98d86c81e2155b1df5eb0c34be66d9
> sekikn@sekikn-ThinkCentre-M715q:~$ docker exec -it 576a624e3e96 bash
> root@576a624e3e96:/# R -q -e 'packageVersion("rmarkdown")'
> > packageVersion("rmarkdown")
> Error in packageVersion("rmarkdown") : package 'rmarkdown' not found
> Execution halted
> root@576a624e3e96:/# R -q -e 'packageVersion("testthat")'
> > packageVersion("testthat")
> Error in packageVersion("testthat") : package 'testthat' not found
> Execution halted
> {code}
> But building these images succeeded with no error on Jenkins:
> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/156/OS=ubuntu-16.04/console
> https://ci.bigtop.apache.org/view/Docker/job/Docker-Toolchain-Trunk/156/OS=ubuntu-16.04-aarch64/console
> {code}
> (snip)
> Notice: /Stage[main]/Bigtop_toolchain::Renv/Exec[install_r_packages]/returns: executed successfully
> (snip)
> BUILD SUCCESSFUL in 22m 17s
> 1 actionable task: 1 executed
> + docker push bigtop/slaves:trunk-ubuntu-16.04
> The push refers to repository [docker.io/bigtop/slaves]
> 459ec69fe443: Preparing
> 694145ca4ac1: Preparing
> eb1493b87b0e: Preparing
> 393364419078: Preparing
> 36855eaee39c: Preparing
> 79560c66832e: Preparing
> d7232280c8c4: Preparing
> 07663827a77f: Preparing
> 87a2d0000622: Preparing
> 4a7a5ec0f29e: Preparing
> 8823818c4748: Preparing
> d7232280c8c4: Waiting
> 07663827a77f: Waiting
> 87a2d0000622: Waiting
> 8823818c4748: Waiting
> 4a7a5ec0f29e: Waiting
> 79560c66832e: Waiting
> 36855eaee39c: Layer already exists
> 79560c66832e: Layer already exists
> d7232280c8c4: Layer already exists
> 07663827a77f: Layer already exists
> 87a2d0000622: Layer already exists
> 4a7a5ec0f29e: Layer already exists
> 8823818c4748: Layer already exists
> 393364419078: Pushed
> 694145ca4ac1: Pushed
> 459ec69fe443: Pushed
> eb1493b87b0e: Pushed
> trunk-ubuntu-16.04: digest: sha256:362379880480575bc4a09bdb8052e01cd364a307df7282165043981752fa7b6f size: 2625
> Finished: SUCCESS
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)