[jira] [Comment Edited] (SPARK-20952) TaskContext should be an InheritableThreadLocal

["Shixiong Zhu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033915#comment-16033915 ] 

Shixiong Zhu edited comment on SPARK-20952 at 6/1/17 11:43 PM:
---------------------------------------------------------------

InheritableThreadLocal only works when creating a new thread. Here you were talking about thread pools. Reusing a thread may get a wrong TaskContext if it's InheritableThreadLocal.


was (Author: zsxwing):
InheritableThreadLocal only works when creating a new thread. Here you were talking about thread pools.

> TaskContext should be an InheritableThreadLocal
> -----------------------------------------------
>
>                 Key: SPARK-20952
>                 URL: https://issues.apache.org/jira/browse/SPARK-20952
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 2.1.1
>            Reporter: Robert Kruszewski
>            Priority: Minor
>
> TaskContext is a ThreadLocal as a result when you fork a thread inside your executor task you lose the handle on the original context set by the executor. We should change it to InheritableThreadLocal so we can access it inside thread pools on executors. 
> See ParquetFileFormat#readFootersInParallel for example of code that uses thread pools inside the tasks.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org