You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Srikanth Sundarrajan <sr...@hotmail.com> on 2015/09/07 11:54:54 UTC
Adding additional jars to distributed cache (yarn-client)
Hi,
Am trying to use JavaSparkContext() to create a new SparkContext and attempted to pass the requisite jars. But looks like they aren't getting added to the distributed cache automatically. Looking into YarnClientSchedulerBackend::start() and ClientArguments, it did seem like it would just add the SPARK_JAR and APP_JAR. Am wondering what is the best way to add additional files to Distributed cache and also have them appear in the classpath for ExecutorLauncher.
Thanks
Srikanth Sundarrajan