[jira] [Resolved] (SPARK-8586) SQL add jar command does not work well with Scala REPL

["Sean Owen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-8586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen resolved SPARK-8586.
------------------------------
    Resolution: Duplicate

I think this and several JIRAs are just the same issue: add jar vs --jars and what classloader ends up with the classes.

> SQL add jar command does not work well with Scala REPL
> ------------------------------------------------------
>
>                 Key: SPARK-8586
>                 URL: https://issues.apache.org/jira/browse/SPARK-8586
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.4.0, 2.1.0
>            Reporter: Yin Huai
>            Assignee: Yin Huai
>            Priority: Critical
>
> Seems SparkIMain always resets the context class loader in {{loadAndRunReq}}. So, SerDe added through add jar command may not be loaded in the context class loader when we lookup the table.
> For example, the following code will fail when we try to show the table. 
> {code}
> hive.sql("add jar sql/hive/src/test/resources/hive-hcatalog-core-0.13.1.jar")
> hive.sql("drop table if exists jsonTable")
> hive.sql("CREATE TABLE jsonTable(key int, val string) ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'")
> hive.createDataFrame((1 to 100).map(i => (i, s"str$i"))).toDF("key", "val").insertInto("jsonTable")
> hive.table("jsonTable").show
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org