You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2018/11/16 01:26:00 UTC
[jira] [Commented] (SPARK-26080) Unable to run worker.py on Windows
[ https://issues.apache.org/jira/browse/SPARK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688885#comment-16688885 ]
Hyukjin Kwon commented on SPARK-26080:
--------------------------------------
We should fix this.
> Unable to run worker.py on Windows
> ----------------------------------
>
> Key: SPARK-26080
> URL: https://issues.apache.org/jira/browse/SPARK-26080
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 2.4.0
> Environment: Windows 10 Education 64 bit
> Reporter: Hayden Jeune
> Priority: Major
>
> Use of the resource module in python means worker.py cannot run on a windows system. This package is only available in unix based environments.
> [https://github.com/apache/spark/blob/9a5fda60e532dc7203d21d5fbe385cd561906ccb/python/pyspark/worker.py#L25]
> {code:python}
> textFile = sc.textFile("README.md")
> textFile.first()
> {code}
> When the above commands are run I receive the error 'worker failed to connect back', and I can see an exception in the console coming from worker.py saying 'ModuleNotFoundError: No module named resource'
> I do not really know enough about what I'm doing to fix this myself. Apologies if there's something simple I'm missing here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org