You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Parth Gandhi (JIRA)" <ji...@apache.org> on 2017/08/31 18:02:00 UTC
[jira] [Comment Edited] (SPARK-21888) Cannot add stuff to Client
Classpath for Yarn Cluster Mode
[ https://issues.apache.org/jira/browse/SPARK-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16149350#comment-16149350 ]
Parth Gandhi edited comment on SPARK-21888 at 8/31/17 6:01 PM:
---------------------------------------------------------------
The spark job runs successfully only if hbase-site.xml is placed in SPARK_CONF_DIR. If I add the xml file to --jars then it gets added to the driver classpath which is required but hbase fails to get a valid Kerberos token as the xml file is not found in the system classpath on the gateway where I launch the application.
was (Author: pgandhi):
The spark job runs successfully only if hbase-site.xml is placed in SPARK_CONF_DIR. If I add the xml file to --jars then it gets added to the driver classmate which is required but hbase fails to get a valid Kerberos token as the xml file is not found in the system classpath on the gateway where I launch the application.
> Cannot add stuff to Client Classpath for Yarn Cluster Mode
> ----------------------------------------------------------
>
> Key: SPARK-21888
> URL: https://issues.apache.org/jira/browse/SPARK-21888
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.2.0
> Reporter: Parth Gandhi
> Priority: Minor
>
> While running Spark on Yarn in cluster mode, currently there is no way to add any config files, jars etc. to Client classpath. An example for this is that suppose you want to run an application that uses hbase. Then, unless and until we do not copy the necessary config files required by hbase to Spark Config folder, we cannot specify or set their exact locations in classpath on Client end which we could do so earlier by setting the environment variable "SPARK_CLASSPATH".
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org