You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by patcharee <Pa...@uni.no> on 2015/03/03 12:08:57 UTC
insert Hive table with RDD
Hi,
How can I insert an existing hive table with an RDD containing my data?
Any examples?
Best,
Patcharee
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: insert Hive table with RDD
Posted by Jagat Singh <ja...@gmail.com>.
Will this recognize the hive partitions as well.
Example
insert into specific partition of hive ?
On Tue, Mar 3, 2015 at 11:42 PM, Cheng, Hao <ha...@intel.com> wrote:
> Using the SchemaRDD / DataFrame API via HiveContext
>
> Assume you're using the latest code, something probably like:
>
> val hc = new HiveContext(sc)
> import hc.implicits._
> existedRdd.toDF().insertInto("hivetable")
> or
>
> existedRdd.toDF().registerTempTable("mydata")
> hc.sql("insert into hivetable as select xxx from mydata")
>
>
>
> -----Original Message-----
> From: patcharee [mailto:Patcharee.Thongtra@uni.no]
> Sent: Tuesday, March 3, 2015 7:09 PM
> To: user@spark.apache.org
> Subject: insert Hive table with RDD
>
> Hi,
>
> How can I insert an existing hive table with an RDD containing my data?
> Any examples?
>
> Best,
> Patcharee
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org For additional
> commands, e-mail: user-help@spark.apache.org
>
>
Re: insert Hive table with RDD
Posted by patcharee <Pa...@uni.no>.
Hi,
I guess that toDF() api in spark 1.3 which is required build from source
code?
Patcharee
On 03. mars 2015 13:42, Cheng, Hao wrote:
> Using the SchemaRDD / DataFrame API via HiveContext
>
> Assume you're using the latest code, something probably like:
>
> val hc = new HiveContext(sc)
> import hc.implicits._
> existedRdd.toDF().insertInto("hivetable")
> or
>
> existedRdd.toDF().registerTempTable("mydata")
> hc.sql("insert into hivetable as select xxx from mydata")
>
>
>
> -----Original Message-----
> From: patcharee [mailto:Patcharee.Thongtra@uni.no]
> Sent: Tuesday, March 3, 2015 7:09 PM
> To: user@spark.apache.org
> Subject: insert Hive table with RDD
>
> Hi,
>
> How can I insert an existing hive table with an RDD containing my data?
> Any examples?
>
> Best,
> Patcharee
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org For additional commands, e-mail: user-help@spark.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
RE: insert Hive table with RDD
Posted by "Cheng, Hao" <ha...@intel.com>.
Using the SchemaRDD / DataFrame API via HiveContext
Assume you're using the latest code, something probably like:
val hc = new HiveContext(sc)
import hc.implicits._
existedRdd.toDF().insertInto("hivetable")
or
existedRdd.toDF().registerTempTable("mydata")
hc.sql("insert into hivetable as select xxx from mydata")
-----Original Message-----
From: patcharee [mailto:Patcharee.Thongtra@uni.no]
Sent: Tuesday, March 3, 2015 7:09 PM
To: user@spark.apache.org
Subject: insert Hive table with RDD
Hi,
How can I insert an existing hive table with an RDD containing my data?
Any examples?
Best,
Patcharee
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org For additional commands, e-mail: user-help@spark.apache.org