[jira] [Created] (SPARK-3008) PySpark fails due to zipimport not able to load the assembly jar (/usr/bin/python: No module named pyspark)

["Jai Kumar Singh (JIRA)" <ji...@apache.org>]

Jai Kumar Singh created SPARK-3008:
--------------------------------------

             Summary: PySpark fails due to  zipimport not able to load the assembly jar (/usr/bin/python: No module named pyspark)
                 Key: SPARK-3008
                 URL: https://issues.apache.org/jira/browse/SPARK-3008
             Project: Spark
          Issue Type: Bug
          Components: PySpark
         Environment: Assemebly Jar target/scala-2.10/spark-assembly-1.1.0-SNAPSHOT-hadoop2.2.0.jar

jar -tf assembly/target/scala-2.10/spark-assembly-1.1.0-SNAPSHOT-hadoop2.2.0.jar | wc -l
70441

git sha commit ba28a8fcbc3ba432e7ea4d6f0b535450a6ec96c6

            Reporter: Jai Kumar Singh


PySpark is not working. It fails because zipimport not able to import assembly  jar because that contain more than 65536 files.


Email chains in this regard are below

http://mail-archives.apache.org/mod_mbox/incubator-spark-user/201406.mbox/%3CCAMJOb8kcGk0PqiOGJu6UoKCeysWCuSW3xwd5wRs8ikpMgD2DAg@mail.gmail.com%3E

https://mail.python.org/pipermail/python-list/2014-May/671353.html


Is there any work around to bypass the issue ?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org