[Apache Livy] Custom jars not being picked up from spark-defaults.conf

[sanchay javeria <sa...@gmail.com>]

Hey,

I defined a bunch of spark configurations
<https://spark.apache.org/docs/latest/configuration.html> in
spark-defaults.conf file. I made sure that the file is present under
$SPARK_HOME/conf/ directory on the Livy host. All of the configs are
getting picked up except spark.jars and I'm really stumped what could be
going wrong.

It works fine when I pass the jars directly with the request object but not
when I add them to the config file. All the jars are coming from S3 file
paths if that's relevant.

Any help would be appreciated!

- Sanchay