You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by satyajit vegesna <sa...@gmail.com> on 2021/12/16 18:04:55 UTC
Add external JARS to classpath not working.
Hi All,
Could someone please advice with my below issues,
Below is my command I am using,
spark-submit --class AerospikeDynamicProtoMessageGenerator --master yarn
--deploy-mode cluster --num-executors 10 --conf
'spark.driver.extraJavaOptions=-verbose:class' --conf
'spark.executor.extraJavaOptions=-verbose:class' --jars
/tmp/sna/accountProfile.jar,/tmp/sna/deviceProfile.jar,/tmp/sna/userProfile.jar
--driver-class-path
/tmp/sna/accountProfile.jar,/tmp/sna/deviceProfile.jar,/tmp/sna/userProfile.jar
--conf
spark.executor.extraClassPath=/tmpsna/accountProfile.jar,/tmp/sna/deviceProfile.jar,/tmp/sna/userProfile.jar
--executor-memory 1G /tmp/sna CodeTest.jar
But i still am unable to access the classes in jars and the way i am trying
to access classes from external JAR is as below,
val clazz = Class.forName("path to the class") and get classnotfounderror.
Regards.