[jira] [Created] (ZEPPELIN-2136) --files in SPARK_SUBMIT_OPTIONS not working

["Meethu Mathew (JIRA)" <ji...@apache.org>]

Meethu Mathew created ZEPPELIN-2136:
---------------------------------------

             Summary: --files in SPARK_SUBMIT_OPTIONS not working 
                 Key: ZEPPELIN-2136
                 URL: https://issues.apache.org/jira/browse/ZEPPELIN-2136
             Project: Zeppelin
          Issue Type: Bug
          Components: pySpark
    Affects Versions: 0.6.0
            Reporter: Meethu Mathew


Acc to the zeppelin documentation, to pass a python package to zeppelin pyspark interpreter, you can export it through --files option in SPARK_SUBMIT_OPTIONS in conf/zeppelin-env.sh. 

When I add a .egg file through the --files option in SPARK_SUBMIT_OPTIONS , zeppelin notebook is not throwing error, but I am not able to import the module inside the zeppelin notebook.

Spark version is 1.6.2 and the zepplein-env.sh file looks like:

    export SPARK_HOME=/home/me/spark-1.6.1-bin-hadoop2.6
    export SPARK_SUBMIT_OPTIONS="--jars /home/me/spark-csv-1.5.0-s_2.10.jar,/home/me/commons-csv-1.4.jar --files /home/me/models/Churn/package/build/dist/fly_libs-1.1-py2.7.egg"

My work around for this problem was to add the .rgg file using sc.addPyFile() inside the notebook.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)