[jira] [Comment Edited] (SPARK-21223) Thread-safety issue in FsHistoryProvider

["zenglinxi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-21223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16066319#comment-16066319 ] 

zenglinxi edited comment on SPARK-21223 at 6/28/17 10:42 AM:
-------------------------------------------------------------

[~sowen] ok, i will check SPARK-21078 first.


was (Author: gostop_zlx):
ok, i will check SPARK-21078 first.

> Thread-safety issue in FsHistoryProvider 
> -----------------------------------------
>
>                 Key: SPARK-21223
>                 URL: https://issues.apache.org/jira/browse/SPARK-21223
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 2.1.1
>            Reporter: zenglinxi
>
> Currently, Spark HistoryServer use a HashMap named fileToAppInfo in class FsHistoryProvider to store the map of eventlog path and attemptInfo. 
> When use ThreadPool to Replay the log files in the list and merge the list of old applications with new ones, multi thread may update fileToAppInfo at the same time, which may cause Thread-safety issues.
> {code:java}
> for (file <- logInfos) {
>        tasks += replayExecutor.submit(new Runnable {
>         override def run(): Unit = mergeApplicationListing(file)
>      })
>  }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org