[jira] [Created] (SPARK-3753) Spark hive join results in empty with shared hive context

["Hector Yee (JIRA)" <ji...@apache.org>]

Hector Yee created SPARK-3753:
---------------------------------

             Summary: Spark hive join results in empty with shared hive context
                 Key: SPARK-3753
                 URL: https://issues.apache.org/jira/browse/SPARK-3753
             Project: Spark
          Issue Type: Bug
    Affects Versions: 1.1.0
            Reporter: Hector Yee
            Priority: Minor


When I have two hive tables and do a join with the same hive context I get the empty set

e.g.

val hc = new HiveContext(sc)
val table1 = hc.sql("SELECT * from t1")
val table2 = hc.sql("SELECT * from t2")
val intersect = table1.join(table2).take(10)
// empty set

but this works if I do 
val hc1 = new HiveContext(sc)
val table1 = hc1.sql("SELECT * from t1")
val hc2 = new HiveContext(sc)
val table2 = hc2.sql("SELECT * from t2")
val intersect = table1.join(table2).take(10)

I am not sure if take is propagating up the take to table1 and table2 and then doing the intersect (in the case of large tables that means no results) or if it is some other problem with hive context.

Doing the join in one SQL query also seems to result in the empty set.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org