[jira] [Updated] (SPARK-1753) PySpark on YARN does not work on assembly jar built on Red Hat based OS

["Patrick Wendell (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-1753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Patrick Wendell updated SPARK-1753:
-----------------------------------

    Assignee: Andrew Or

> PySpark on YARN does not work on assembly jar built on Red Hat based OS
> -----------------------------------------------------------------------
>
>                 Key: SPARK-1753
>                 URL: https://issues.apache.org/jira/browse/SPARK-1753
>             Project: Spark
>          Issue Type: Bug
>          Components: PySpark, Spark Core
>    Affects Versions: 1.0.0
>            Reporter: Andrew Or
>            Assignee: Andrew Or
>             Fix For: 1.0.0
>
>
> If the jar is built on a Red Hat based OS, the additional python files included in the jar cannot be accessed. This means PySpark doesn't work on YARN because in this mode it relies on the python files within this jar.
> I have confirmed that my Java, Scala, and maven versions are all exactly the same on my CentOS environment and on my local OSX environment, and the former does not work. Thomas Graves also struggled with the same problem.
> Until a fix is found, we should at the very least document this peculiarity.



--
This message was sent by Atlassian JIRA
(v6.2#6252)