[jira] [Created] (SPARK-10436) spark-submit overwrites spark.files defaults with the job script filename

["axel dahl (JIRA)" <ji...@apache.org>]

axel dahl created SPARK-10436:
---------------------------------

             Summary: spark-submit overwrites spark.files defaults with the job script filename
                 Key: SPARK-10436
                 URL: https://issues.apache.org/jira/browse/SPARK-10436
             Project: Spark
          Issue Type: Bug
          Components: Spark Submit
    Affects Versions: 1.4.0
         Environment: Ubuntu, Spark 1.4.0 Standalone
            Reporter: axel dahl
            Priority: Minor


In my spark-defaults.conf I have configured a set of libararies to be uploaded to my Spark 1.4.0 Standalone cluster.  The entry appears as:

spark.files              libarary.zip,file1.py,file2.py

When I execute spark-submit -v test.py

I see that spark-submit reads the defaults correctly, but that it overwrites the "spark.files" default entry and replaces it with the name if the job script, i.e. "test.py".

This behavior doesn't seem intuitive.  test.py, should be added to the spark working folder, but it should not overwrite the "spark.files" defaults.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org