You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/06/05 06:51:10 UTC

[GitHub] [spark] pquentin edited a comment on pull request #22480: [SPARK-25473][PYTHON][SS][TEST] ForeachWriter tests failed on Python 3.6 and macOS High Sierra

pquentin edited a comment on pull request #22480:
URL: https://github.com/apache/spark/pull/22480#issuecomment-638786759


   @HyukjinKwon This continues to be a problem for us, as we tend to forget to use the workaround in new projects.
   
   As noted in https://github.com/mozilla/mozanalysis/issues/40#issuecomment-495807665, I think the root cause is that [PySpark calls os.fork()](https://github.com/apache/spark/blob/3b14088/python/pyspark/daemon.py#L147) in a way that should simply be avoided on macOS: https://www.evanjones.ca/fork-is-dangerous.html


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org