[jira] [Resolved] (SPARK-1718) pyspark doesn't work with assembly jar containing over 65536 files/dirs built on redhat

["Thomas Graves (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Thomas Graves resolved SPARK-1718.
----------------------------------

    Resolution: Duplicate

> pyspark doesn't work with assembly jar containing over 65536 files/dirs built on redhat 
> ----------------------------------------------------------------------------------------
>
>                 Key: SPARK-1718
>                 URL: https://issues.apache.org/jira/browse/SPARK-1718
>             Project: Spark
>          Issue Type: Bug
>          Components: PySpark
>    Affects Versions: 1.0.0
>            Reporter: Thomas Graves
>
> Recently pyspark was ported to yarn (pr 30), but when I went to try it I couldn't get it work.  I was building on a redhat 6 box.  I figured out that if the assembly jar file contained over 65536 files/directories then it wouldn't work.  If I unjarred the assembly and removed some stuff to get it under 65536 and jarred it back up, then it would work.     
> It appears to only be an issue when building on a redhat box as I can build on my mac and it works just fine there.   



--
This message was sent by Atlassian JIRA
(v6.2#6252)