You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "shanyu zhao (JIRA)" <ji...@apache.org> on 2018/11/10 00:05:00 UTC
[jira] [Commented] (SPARK-25999) make-distribution.sh failure with
--r and -Phadoop-provided
[ https://issues.apache.org/jira/browse/SPARK-25999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682089#comment-16682089 ]
shanyu zhao commented on SPARK-25999:
-------------------------------------
patch attached. Basically it creates an optional project that brings all dependencies to R/rjarsdep/target folder, and copy the missing jars to assembly/target folder before building R.
> make-distribution.sh failure with --r and -Phadoop-provided
> -----------------------------------------------------------
>
> Key: SPARK-25999
> URL: https://issues.apache.org/jira/browse/SPARK-25999
> Project: Spark
> Issue Type: Bug
> Components: Build
> Affects Versions: 2.3.2, 2.4.0
> Reporter: shanyu zhao
> Priority: Major
> Attachments: SPARK-25999.patch
>
>
> It is not possible to build a distribution that doesn't contain hadoop dependencies but include SparkR. This is because R/check_cran.sh builds R document which depends on hadoop dependencies in assembly/target/scala-xxx/jars folder.
> To reproduce:
> MAVEN_BUILD_OPTS="-Dmaven.javadoc.skip=true -Pyarn -Phadoop-2.7 -Phive -Psparkr -Phadoop-provided"
> ./dev/make-distribution.sh --tgz --r $MAVEN_BUILD_OPTS
>
> Error:
> * creating vignettes ... ERROR
> ...
> Error: A JNI error has occurred, please check your installation and try again
> Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/Logger
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org