You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by pth001 <Pa...@uni.no> on 2015/06/13 21:36:09 UTC

Dataframe Write : Tables created with SQLContext must be TEMPORARY. Use a HiveContext instead.

Hi,

I am using spark 0.14. I try to insert data into a hive table (in orc 
format) from DF.

partitionedTestDF.write.format("org.apache.spark.sql.hive.orc.DefaultSource")
.mode(org.apache.spark.sql.SaveMode.Append).partitionBy("zone","z","year","month").saveAsTable("testorc")

When this job is submitted by spark-submit I get >>
Exception in thread "main" java.lang.RuntimeException: Tables created 
with SQLContext must be TEMPORARY. Use a HiveContext instead

But the job works fine on spark-shell. What can be wrong?

BR,
Patcharee

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Re: Dataframe Write : Tables created with SQLContext must be TEMPORARY. Use a HiveContext instead.

Posted by Cheng Lian <li...@gmail.com>.
As the error message says, were you using a |SQLContext| instead of a 
|HiveContext| to create the DataFrame?

In Spark shell, although the variable name is |sqlContext|, the type of 
that variable is actually |org.apache.spark.sql.hive.HiveContext|, which 
has the ability to communicate with Hive metastore.

Cheng

On 6/13/15 12:36 PM, pth001 wrote:

> Hi,
>
> I am using spark 0.14. I try to insert data into a hive table (in orc 
> format) from DF.
>
> partitionedTestDF.write.format("org.apache.spark.sql.hive.orc.DefaultSource") 
>
> .mode(org.apache.spark.sql.SaveMode.Append).partitionBy("zone","z","year","month").saveAsTable("testorc") 
>
>
> When this job is submitted by spark-submit I get >>
> Exception in thread "main" java.lang.RuntimeException: Tables created 
> with SQLContext must be TEMPORARY. Use a HiveContext instead
>
> But the job works fine on spark-shell. What can be wrong?
>
> BR,
> Patcharee
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
> .
>
​