You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Saisai Shao (JIRA)" <ji...@apache.org> on 2017/09/01 09:13:00 UTC
[jira] [Commented] (SPARK-21888) Cannot add stuff to Client
Classpath for Yarn Cluster Mode
[ https://issues.apache.org/jira/browse/SPARK-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16150258#comment-16150258 ]
Saisai Shao commented on SPARK-21888:
-------------------------------------
Jars added by "--jars" will be added to client classpath in yarn-cluster mode.
In your case the only problem is about hbase-site.xml, normally we will put this file in SPARK_CONF_DIR as well as hive-site.xml, doesn't it work for your?
> Cannot add stuff to Client Classpath for Yarn Cluster Mode
> ----------------------------------------------------------
>
> Key: SPARK-21888
> URL: https://issues.apache.org/jira/browse/SPARK-21888
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.2.0
> Reporter: Parth Gandhi
> Priority: Minor
>
> While running Spark on Yarn in cluster mode, currently there is no way to add any config files, jars etc. to Client classpath. An example for this is that suppose you want to run an application that uses hbase. Then, unless and until we do not copy the necessary config files required by hbase to Spark Config folder, we cannot specify or set their exact locations in classpath on Client end which we could do so earlier by setting the environment variable "SPARK_CLASSPATH".
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org