You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by satyajit vegesna <sa...@gmail.com> on 2021/12/16 18:04:55 UTC

Add external JARS to classpath not working.

Hi All,

Could someone please advice with my below issues,
Below is my command I am using,

spark-submit --class AerospikeDynamicProtoMessageGenerator --master yarn
--deploy-mode cluster --num-executors 10 --conf
'spark.driver.extraJavaOptions=-verbose:class' --conf
'spark.executor.extraJavaOptions=-verbose:class' --jars
/tmp/sna/accountProfile.jar,/tmp/sna/deviceProfile.jar,/tmp/sna/userProfile.jar
--driver-class-path
/tmp/sna/accountProfile.jar,/tmp/sna/deviceProfile.jar,/tmp/sna/userProfile.jar
--conf
spark.executor.extraClassPath=/tmpsna/accountProfile.jar,/tmp/sna/deviceProfile.jar,/tmp/sna/userProfile.jar
--executor-memory 1G /tmp/sna CodeTest.jar

But i still am unable to access the classes in jars and the way i am trying
to access classes from external JAR is as below,

val clazz = Class.forName("path to the class") and get classnotfounderror.

Regards.