[jira] [Commented] (SPARK-2498) Concurrent initialization of various DataType objects causes exceptions

["Zongheng Yang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062858#comment-14062858 ] 

Zongheng Yang commented on SPARK-2498:
--------------------------------------

Github PR: https://github.com/apache/spark/pull/1423

> Concurrent initialization of various DataType objects causes exceptions
> -----------------------------------------------------------------------
>
>                 Key: SPARK-2498
>                 URL: https://issues.apache.org/jira/browse/SPARK-2498
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.0.1
>            Reporter: Zongheng Yang
>            Priority: Critical
>
> Reported by Keith Simmons and gorenuru [here|http://apache-spark-user-list.1001560.n3.nabble.com/Error-while-running-Spark-SQL-join-when-using-Spark-1-0-1-td9776.html]. This is probably due to the fact that Scala Reflection API is not thread-safe for 2.10.* versions; see [SI-6240|https://issues.scala-lang.org/browse/SI-6240] for more details.
> The purposed fix is to add a lock object that is synchronized on when various data type objects make use of the reflection API (during their initializations or other places in program execution). (Longer-term, we could consider evolving this to a SparkSQL-global lock, and all calls to the reflection APi from Spark SQL should probably synchronize on it.)



--
This message was sent by Atlassian JIRA
(v6.2#6252)