[jira] [Updated] (SPARK-8586) SQL add jar command does not work well with Scala REPL

["Yin Huai (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-8586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yin Huai updated SPARK-8586:
----------------------------
    Target Version/s: 1.5.0, 1.4.2  (was: 1.4.1, 1.5.0, 1.4.2)

> SQL add jar command does not work well with Scala REPL
> ------------------------------------------------------
>
>                 Key: SPARK-8586
>                 URL: https://issues.apache.org/jira/browse/SPARK-8586
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.4.0
>            Reporter: Yin Huai
>            Assignee: Yin Huai
>            Priority: Critical
>
> Seems SparkIMain always resets the context class loader in {{loadAndRunReq}}. So, SerDe added through add jar command may not be loaded in the context class loader when we lookup the table.
> For example, the following code will fail when we try to show the table. 
> {code}
> hive.sql("add jar sql/hive/src/test/resources/hive-hcatalog-core-0.13.1.jar")
> hive.sql("drop table if exists jsonTable")
> hive.sql("CREATE TABLE jsonTable(key int, val string) ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'")
> hive.createDataFrame((1 to 100).map(i => (i, s"str$i"))).toDF("key", "val").insertInto("jsonTable")
> hive.table("jsonTable").show
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org