[jira] [Commented] (SPARK-14686) Implement a non-inheritable localProperty facility

["Josh Rosen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15244478#comment-15244478 ] 

Josh Rosen commented on SPARK-14686:
------------------------------------

The idea of making these properties non-inheritable has come up in the past because of other problems that this inheritance has caused. Here are a few relevant JIRA tickets:

- https://issues.apache.org/jira/browse/SPARK-12964
- https://issues.apache.org/jira/browse/SPARK-10407
- https://issues.apache.org/jira/browse/SPARK-10563
- https://issues.apache.org/jira/browse/SPARK-6629

We should try to summarize all of these to figure out whether we can just remove the inheritance.

> Implement a non-inheritable localProperty facility
> --------------------------------------------------
>
>                 Key: SPARK-14686
>                 URL: https://issues.apache.org/jira/browse/SPARK-14686
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>            Reporter: Marcin Tustin
>            Priority: Minor
>
> As discussed here: http://mail-archives.apache.org/mod_mbox/spark-dev/201604.mbox/%3CCANXtaKA4ZdpiUbZPnzDBN8ZL7_RwKSMuz6n45ixnHVEBE1hgjg%40mail.gmail.com%3E
> Spark localProperties are always inherited by spawned threads. There are situations in which this is undesirable (notably spark.sql.execution.id and any other localProperty that should always be cleaned up). This is a ticket to implement a non-inheritable mechanism for localProperties. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org